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AN  OVERVIEW  OF  THE  AIR  TRAFFIC  CONTROL  SPECIALIST  AND 
FIRST-LINE  SUPERVISOR  SELECTION  SYSTEMS 

Jennifer  G.  Myers,  Ph.D. 


The  air  traffic  control  (ATC)  job  is  unique  in 
several  ways.  It  is  one  of  the  few  federal  jobs 
where  continued  employment  is  based  on  the 
successful  completion  of  a  selection  and  training 
program  wholely  conducted  by  the  government. 
The  skills  necessary  to  perform  the  job  are  not 
necessarily  those  acquir^  through  formal  voca¬ 
tional  or  university  training,  although  university- 
based  ATC  training  has  recently  been  initiated  on 
a  test  basis.  Like  other  technical  occupations, 
reaching  journeyman  status  is  based  on  the 
successive  demonstration  of  the  acquisition  of 
increasingly  complex  job  skills. 

Individuals  interested  in  applying  for  an  ATC 
job  first  complete  a  battery  of  tests  administered 
by  the  Office  of  Personnel  Management  in 
different  regional  locations.  Eligibility  for  attend¬ 
ing  the  FAA  Academy  Screen  Program  in 
Oklahoma  City  is  based  on  a  composite  of  the 
test  battery  scores,  as  well  as  meeting  medical 
and  security  screening  criteria.  Following  the 
successful  completion  of  the  Screen  Program,  in¬ 
dividuals  are  assigned  to  a  field  facility  for  on- 
the-job  training.  The  length  of  field  training 
varies,  dqiending  on  the  type  of  facility  (en  route 
or  terminal),  and  ranges  from  about  1  to  3  years 
(see  Figure  1). 

Air  traffic  control  specialists  (ATCS)  who 
wish  to  advance  into  a  supervisory  position  must 
apply  to  the  Supervisory  Identification  and 
Development  Program  (SIDP).  This  is  an  agen¬ 
cy-developed  system  for  assessing  individual 
competencies  and  determining  eligibility  to 
compete  for  first-line  supervisory  positions.  The 
SIDP  was  first  proposed  in  response  to  the 
recommendations  of  congressiomd  committees 
following  the  air  traffic  controller  strike  in  1981. 
It  was  felt  that  by  basing  supervisory  selections 
on  factors  other  tfian  prior  training  and  technical 
performance,  the  organizational  culture  that 
precipitated  the  1981  strike  could  be  improved 
and  frture  labor-management  problems  could  be 
avoided.  The  SIDP  was  implemented  on  a  test 
basis  in  2  regions.  Southwest  and  Northwest 


Mountain,  in  1985  and  continued  until  national 
implementation  of  the  program  in  1988.  Since 
1985,  over  10,000  supervisors  and  non-super- 
visors  have  applied  to  the  program  and  approxi¬ 
mately  5,000  of  those  applicants  have  successful¬ 
ly  completed  the  program. 

Continued  technical  competence  in  air  traffic 
control  is  required  in  the  supervisory  position 
because  task  performance  includes  supervision  of 
the  technical  work  of  others.  This  entails  a 
requirement  for  maintaining  operational  currency 
by  performing  in  an  operational  position  16 
hours  per  month.  However,  technical  competence 
is  but  one  of  the  competencies  assessed  in  the 
SIDP.  Rather,  the  SIDP  is  designed  to  allow 
^plicants  to  demonstrate  other  skills  needed  for 
effective  supervision  of  people  and  programs, 
such  as  conununications  and  interpersonal  skills. 
The  SIDP  is  a  multiple  hurdle  process  for  deter¬ 
mining  eligibility  for  supervisory  positions  (see 
Figure  2). 

Controllers  must  have  completed  one  year  at 
the  journeyman  level  and  received  a  fully  suc¬ 
cessful  rating  on  their  last  performance  appraisal 
to  be  eligible  to  apply  to  the  program.  The  Peer- 
Supervisory  Assessment  process  requires  the 
SIDP  applicant  to  identify  from  4  to  7  peers 
(d^ending  on  the  size  of  the  facility)  and  one  or 
more  supervisors  to  rate  him/her  on  4  perfor¬ 
mance  dimensions:  Interpersonal,  Communica¬ 
tions,  Direction  and  Motivation,  and  Technical 
Competence.  A  paired-comparison  method  is 
used  to  rate  each  applicant  against  other  ap¬ 
plicants  and  benchmarks  on  each  of  the  dimen¬ 
sions.  Supervisors  and  peer  ratings  are  combined 
to  produce  a  single  score  for  each  dimension. 
Dimension  ratings  are  averaged  to  create  a 
composite  score  and  the  top  applicants  (based  on 
their  composite  score)  are  referred  to  the  next 
stage  of  the  selection  process,  the  Skill-Based 
Interview  (SBI). 

The  SBI  is  a  combination  of  a  face-to-face 
interview  and  role  play  exercises.  In  the  face-to- 
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face  interview,  the  candidate  is  asked  8  questions 
to  assess  Organizational  Knowledge  and  Know¬ 
ledge  of  the  Supervisory  Role.  The  role-plays 
are  designed  to  simulate  realistic  situations  en¬ 
countered  as  a  supervisor.  Based  on  their  perfor¬ 
mance  in  the  scenarios,  applicants  are  rat^  on  7 
competencies  (Problem  Solving  and  Analytical 
Ability,  Judgment,  Decisiveness,  Organizing  and 
Planning  Ability,  Interpersonal  skill,  Communi¬ 
cation  Skill,  and  Direction  and  Motivation)  by  a 
trained  panel  of  3  interviewers  and  receive  verbal 
and  written  feedback  following  the  interview.  A 
description  of  the  competencies  are  contained  in 
Appendix  A. 

There  are  3  possible  direct  outcomes  from 
the  SBI:  referral  to  (a)  self-development,  (b)  the 
Candidate  Review  Board,  or  (c)  the  Eligible  for 
Consideration  list.  Those  who  demonstrate 
ineffective  performance  in  the  SBI  are  counseled 
on  their  strengths  and  weaknesses  and  reconunen- 
ded  for  self-development  prior  to  reapplying  to 
the  SIDP.  Those  who  perform  at  a  moderate 
level  are  referred  to  a  Candidate  Review  Board. 
The  board  d^ermines  whether  the  individual  will 
be  placed  in  formal  development  for  skill  remed¬ 
iation  or  will  be  recommended  for  self-develop¬ 
ment  activities  before  reapplying  to  a  future 
announcement  of  the  SIDP.  Those  who  are 
assigned  to  formal  devdopment  are  provided 
resources  for  receiving  training  and/or  additional 
experiences  (e.g.,  details  or  "shadowing"  assign¬ 
ments)  that  will  remediate  skill  deficiencies  and 
result  in  eligibility  for  supervisory  positions. 
Applicants  who  perform  at  a  high  level  on  each 
of  foe  competencies  are  placed  on  an  Eligible  for 
Consideration  list  and  are  referred  to  position 
vacancies. 

Individuals  who  are  on  foe  eligible  for  con¬ 
sideration  list  are  polled  for  their  interest  in 
applying  for  vacant  supervisory  positions.  Once 
a  list  of  eligible  and  interested  bidders  is  es¬ 
tablished,  foe  list  is  sent  to  foe  selecting  officials 
along  with  a  packet  of  information  containing  foe 
PSA  and  SBI  results  and  work  history  informa¬ 
tion  (awards,  training,  and  job  experience)  for 
each  individual  referred  to  foe  vacancy. 


advancement  and  recognition  in  nonsupervisory 
positions  is  based  on  foe  achievement  of  a  high 
level  of  technical  competence.  However,  for 
promotion  to  a  first-line  supervisor  position, 
ATCSs  must  demonstrate  not  only  technical 
competence  but  also  skills  in  areas  such  as 
decision-making,  communication,  and  leadership. 
Several  questions  arise  in  foe  selection  of  super¬ 
visors  from  a  technical  workforce:  Given  that 
technical  competence  is  required  to  perform  foe 
first-line  supervisor  job,  does  prior  technical 
training  and  journeyman  performance  relate  to 
"promotability”  or  actual  selection  for  a  super¬ 
visor  position?  What  characteristics  distinguish 
those  who  are  selected  from  foe  eligible  pool 
from  those  that  are  not  selected?  These  questions 
are  addressed  in  foe  following  papers,  which 
examine  ATCS  entry  level,  screening,  and 
training  performance  in  relation  to  SIDP  perfor¬ 
mance. 


As  demonstrated  in  foe  description  of  foe 
ATCS  selection  and  training  program,  individual 
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APPENDIX  A 

FIRST-LINE  SUPERVISOR  COMPETENCIES 


ORGANIZATIONAL  KNOWLEDGE:  Demon¬ 
strates  knowledges  of  the  FAA  organizational 
components,  the  mission(s)  of  each  relevant 
organizational  unit,  and  the  principal  programs  in 
the  FAA. 

KNOWLEDGE  OF  SUPERVISORY  ROLE 
PERFORMANCE:  Displays  knowledge  of  the 
roles,  responsibilities,  and  duties  of  supervisors 
and  managers;  accurately  assesses  the  impact 
upon  others  of  role  performance;  and  supports 
a^  promotes  organizational  decisions,  policies, 
programs  and  initiatives  such  as  EEO,  Employee 
Assistance  Program,  Survey-Feedback-Action 
Program,  and  Affirmative  Aaion. 

COMMUNICATION  SKILL:  Presents  and 
expresses  ideas  and  information  effectively  and 
concisely  in  an  oral  and/or  written  mode;  listens 
and  comprdbends  what  others  are  saying;  shares 
information  with  others  and  facilitates  the  open 
exchange  of  ideas  and  information;  is  open, 
honest,  and  straightforward  with  oth»s;  provides 
a  complete  and  timely  explanation  of  issues  and 
decisions  in  a  manner  appropriate  for  the  aud¬ 
ience;  and  present  information  and  material  in  a 
manner  which  gains  the  agreement  of  others. 

DECISIVENESS:  Makes  decisions,  renders 
judgments,  and  takes  action  on  difficult  or  un¬ 
pleasant  tasks  in  a  timely  fashion,  to  include  the 
appropriate  communication  of  both  negative  and 
positive  information  and  decisions. 

DIRECTION  AND  MOTIVATION:  Motivates 
and  provides  direction  in  the  activities  of  others 
to  accomplish  goals;  gains  the  respect  and  con¬ 
fidence  of  others;  appropriately  assigns  work  and 
authority  to  others  in  the  accomplishment  of 
goals;  provides  advice  and  assistance  as  required; 
and  establishes  high  quality  work  standards  for 
self  and  others. 

TECHNICAL  COMPETENCE:  Understands 
and  appropriately  ^plies  procedures,  require¬ 
ments,  regulations,  and  policies;  maintains 
credibility  with  others  on  technical  matters;  and 
uses  equipment,  procedures,  or  systems  in  the 


operational  and/or  staff  environment  as  the 
position  requires. 

INTERPERSONAL  SKILL:  Is  aware  of,  re¬ 
sponds  to,  and  considers  the  needs,  feelings,  and 
csqiabilities  of  others;  deals  effectively  with 
ofoers  in  both  favorable  aM  unfavorable  situa¬ 
tions  regardless  of  their  status  or  position;  ac- 
cqpts  int^personal  and  cultural  differences; 
manages  conflicts,  confrontations,  and  disagree¬ 
ments  in  a  positive  manner  which  minimizes 
personal  impact,  to  include  controlling  one’s  own 
fedings  and  reactions;  and  provides  appropriate 
support  to  others. 

JUDGMENT:  Devdops  and  evaluates  alternative 
courses  of  action;  makes  decisions  based  on 
correct  assumptions  concerning  resources  and 
guidelines;  supports  decisions  or  recommenda¬ 
tions  with  data  or  reasoning;  defines  and  imple¬ 
ments  solutions  to  problems;  and  recognizes 
when  no  action  is  required. 

PLANNING  AND  ORGANIZING:  Identifies 
requirements,  allocates,  and  effectively  uses 
information,  personnel,  time,  and  other  resources 
necessary  for  mission  accomplishment;  es¬ 
tablishes  appropriate  courses  of  action  for  sdf 
and/or  others  to  accomplish  specific  goals; 
develops  evaluation  criteria  and  tracking  systems 
for  monitoring  goal  progress  and  accomplish¬ 
ment;  and  specifies  objectives,  schedules,  and 
priorities. 

PROBLEM  SOLVING  AND  ANALYTICAL 
ABILITY:  Identifies  existing  and  potential 
problems;  notes,  understands  and  includes  the 
critical  elements  of  problem  situations;  obtains 
and  evaluates  relevant  information;  demonstrates 
awareness  that  new  and/or  additional  information 
sources  are  required;  notes  interrelationships 
among  elements;  identifies  piossible  causes  of  the 
problems;  recognizes  the  need  to  shift  to  an 
dtemative  course  of  action  including  innovative 
or  creative  approaches;  and  appropriately  ter¬ 
minates  information  collection  and  evaluation 
activities. 
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COGNITIVE  INDICATORS  OF  ATCS  TECHNICAL  ABILITY  AND 
PERFORMANCE  IN  A  SUPERVISORY  SELECTION  PROGRAM 

David  J.  Schroeder,  Ph.D. 


Historically,  first  level  supervisors  have  been 
selected  with  respect  to  their  technical  com¬ 
petence  and  their  performance  on  the  job  they 
will  supervise.  Raza  (1987),  in  a  comprdiensive 
review  of  the  literature  concerning  the  selection 
of  first  level  supervisors,  concludes  that  while 
supervisors,  researchers,  and  human  resource 
personnel  have  for  a  number  of  years  felt  that 
interpersonal  skills  are  critically  important  factors 
in  the  performance  of  first-level  supervisors, 
current  selection  procedures  for  those  positions 
have,  by  and  large,  continued  to  emphasize 
technical  competence.  Furthermore,  Raza  (1987) 
provides  convincing  evidence  that  current  pro¬ 
cedures  are  less  than  optimal.  The  Supervisory 
Identification  and  Development  Program  (SIDP), 
developed  by  the  FAA,  is  designed  to  incor¬ 
porate  information  other  than  technical  perfor¬ 
mance  of  the  applicant  in  the  selection  process 
for  supervisory  Air  Traffic  Control  Specialists 
(ATCSs). 

Applicants  to  the  ATCS  job  must  first  com¬ 
plete  a  2-stage  selection  process;  a  battery  of 
selection  tests  administered  by  the  Office  of 
Personnel  Management  (0PM)  and  a  9  week, 
performance-based  Screen  program  at  the  FAA 
Academy  in  Oklahoma  City.  This  investigation 
was  developed  to  determine  the  relationship 
between  entry  level  information  on  t^plicants’ 
cognitive  aptitudes  for  performing  ATCS  work, 
as  measured  by  tests  in  the  0PM  selection  bat¬ 
tery,  and  subsequent  selection  as  a  potential 
supervisor  through  the  SIDP.  Specifically,  the 
interest  was  in  determining  if  scores  on  the  initial 
selection  tests  predict  technical  performance 
ratings  received  by  the  SIDP  applicants.  This 
analysis  was  undertaken  in  light  of  the  previous 
research  (Raza,  1987)  that  suggested  that  existing 
procedures  for  supervisory  positions  still  tend  to 
emphasize  technical  competence.  If  that  remains 
true  for  the  SIDP  process,  entry  level  measures 
of  technical  aptitude  may  well  predict  subsequent 
SIDP  performance. 


METHOD 

Subjects 

The  0PM  selection  battery  is  administered  to 
all  applicants  to  determine  hiring  eligibility. 
0PM  scores  were  entered  into  the  analytic  data 
base  for  2,493  ATCSs  who  iqiplied  to  SIDP 
between  1985  and  1989.  Of  this  group,  data  were 
available  for  1,215  who  entered  into  the  en  route 
option  and  1,084  in  the  terminal  option.  This 
division  was  based  on  the  nature  of  the  Academy 
Screen  program  (terminal  or  en  route)  and  the 
type  of  facility  to  which  the  ATCSs  were  subse¬ 
quently  assigned  following  completion  of  the 
Academy  Screen.  The  2  types  of  facilities  do 
involve  significant  differences  in  the  overall 
complexity  and  type  of  air  traffic  they  control.  A 
small  sample  of  en  route  and  terminal  Academy 
entrants  also  completed  2  experimental  tests  -  the 
Directional  Headings  Test  DHT;  en  route 
N=238,  terminal  N=  136  and  Dial  Reading  Test 
(DRT:  en  route  N=139,  terminal  N=134). 

Measures 

0PM  Seleaion  Battery.  The  current  0PM 
selection  battery  was  implemented  in  1981  (see 
Sells,  Dailey  and  Pickrel,  1984  for  information 
concerning  the  validation  of  the  test  battery)  and 
is  comprised  of  the  Multiplex  Controller  Aptitude 
Test  (MCAT),  the  Abstract  Reasoning  Test 
(ABSR),  and  the  Occupational  Knowledge  Test 
(OKT).  The  MCAT  is  a  speeded  paper  and 
pencil  test  that  presents  a  simulated  controller 
work  sample.  A  m^  containing  several  aircraft 
on  various  air  routes  is  presented,  along  with 
tabular  data  concerning  foe  aircrafts’  altitude, 
speed,  and  route  for  each  question.  Aircraft 
positions  and  interpretation  of  foe  tabular  data  are 
required  to  resolve  questions  concerning  time  and 
distance  and  possible  conflicts  between  aircraft 
(see  Figure  1).  Harris  (1986),  in  a  study  relating 


MCAT  scores  to  a  battery  of  cognitive  tests, 
concluded  that  the  MCAT  measures  "both  per- 
c^tual  and  cognitive  reasoning  abilities."  This 
result  was  consistent  with  the  research  findings  as 
reported  in  Sells,  Dailey,  and  Pickrel  (1984). 

The  ABSR  consists  of  a  series  of  figures  or 
letters  that  illustrate  principles  of  logic.  The 
objective  is  to  select  from  several  alternatives  the 
one  that  correctly  completes  a  sequence.  The 
ABSR  measures  aspects  of  logical  reasoning  and 
spatial-perceptual  ability  (see  Figure  2).  The 
OKT,  a  test  of  job  knowledge,  consists  of  ques¬ 
tions  about  air  traffic  control  phraseology  and 
procedures,  air  navigation,  and  aviation  weather. 
A  transformed  score,  referred  to  as  the  Trans¬ 
muted  Composite  (TMC),  is  derived  by  weight¬ 
ing  the  MCAT  and  ABSR  to  produce  a  distribu¬ 
tion  with  a  mean  of  70  and  a  maximum  score  of 
100.  The  final  0PM  rating  includes  the  addition 
of  extra  points  for  the  applicant’s  OKT  score  and 
v^eran’s  preference  points.  0PM  selection 
battery  performance  measures  for  this  study 
included  the  raw  scores  for  the  MCAT,  ABSR, 
and  OKT  and  the  TMC. 

Dial  Reading  and  Directional  Heading  Tests. 
The  DRT  and  DHT  are  2  experimental  tests  that 
were  completed  by  a  small  sample  of  the  SIDP 
applicant  group,  llie  DRT  is  a  brief  timed  test  in 
which  individuals  are  asked  to  identify  and 
correctly  interpret  seven  different  instrument 
dials  for  a  series  of  questions  (see  Figure  3).  The 
DRT  was  developed  in  the  1950’s  for  the  US  Air 
Force  to  select  candidates  for  undergraduate  pilot 
training.  Research  by  Boone  (1979),  Marshall- 
Mies  and  Colmen  vl976)  and  Schroeder,  Dollar 
and  Nye  (1990)  has  demonstrated  the  utility  of 
the  DRT  in  predicting  success  of  ATCS  trainees. 
The  DHT  is  a  speeded  paper  and  pencil  test  of 
spatial  ability.  Each  item  is  comprised  of  3  bits 
of  information  that  reflect  the  cardinal  points  on 
a  compass  (e.g.,  the  letter  "W",  a  symbol  "  ", 

and  the  notation  "270",  each  denote  "West",  see 
Figure  4).  Other  letters,  symbols,  and  notations 
refer  to  the  other  cardinal  points  on  a  compass 
(North,  East  or  South).  In  the  first  of  2  parts  of 
the  DHT,  the  individual  is  asked  to  determine  the 
direction  indicated  (if  all  information  is  consis¬ 
tent),  or  indicate  that  the  information  is  inconsis¬ 
tent.  In  the  second  part  of  the  test,  the  direction 


can  be  determined  if  2  or  more  of  the  3  types  of 
information  presented  agree;  the  direction  cannot 
be  determined  when  all  3  types  of  information 
disagree.  Research  has  also  demonstrated  that  the 
DHT  can  be  used  successfully  to  select  personnel 
for  ATCS  training  (Cobb  &  Matthews,  1972; 
Boone,  1979;  Schroeder,  Dollar  &  Nye,  1990). 

SIDP  Performance.  The  SIDP  process  was 
described  in  the  introduction  by  Myers  (1992). 
PSA  ratings  used  in  this  investigation  include:  the 
composite  (COMP);  conununication  (COMM); 
interpersonal  skills  (INTR);  leadership  (LEAD); 
and  technical  competence  (TECH).  Rating  results 
from  the  Skill-Based  Interview  were  summed  to 
create  an  overall  score  (SBI  TOTAL). 

Analyses.  Means  and  standard  deviations  for 
the  0PM  selection  battery  were  obtained  for  the 
initial  ATCS  applicant  group,  entrants  to  the 
FAA  ATCS  Screen  program,  those  who  success¬ 
fully  completed  the  ATCS  Screen  program,  SIDP 
s^plicants,  SIDP  candidates  placed  on  the  list  of 
eligibles,  and  eligibles  select^  to  fill  supervisory 
positions.  Separate  analyses  were  completed  for 
ATCSs  in  the  terminal  and  en  route  options. 

Correlations  among  0PM  selection  tests, 
DHT,  and  DRT  with  the  PSA  and  SBI  com¬ 
ponents  of  the  SIDP  were  computed.  The  ana¬ 
lyses  were  conducted  separately  for  the  2  op¬ 
tions. 

RESULTS 

Means  and  standard  deviations  of  scores  on 
each  component  of  the  0PM  test  battery  for  en 
route  and  terminal  option  ATCSs  are  presented 
in  Table  1.  The  effects  of  restriction  in  range 
that  occur  as  part  of  the  selection  process  are 
evident  in  the  higher  MCAT  scores  of  Academy 
entrants  who  subsequently  enter  the  en  route 
option  (En  Route  M=90.0,  Terminal  M  =  88.5 
versus  M=69.7  for  all  applicants)  and  the  cor¬ 
responding  smaller  standard  deviations  (SD  of 
7.1  in  en  route;  7.3  terminal  versus  16.1  for 
applicants).  These  differences  were  also  mirrored 
in  the  ABSR  scores  for  both  options.  The  lack  of 
evidence  of  any  restriction  in  range  for  the  OKT 
is  due,  in  part,  to  the  small  number  of  applicants 
who  actually  have  an  ATC  background.  With  the 
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excq>tion  of  <t  lower  average  MCAT  score  for 
those  selected  as  supervisors  in  the  terminal 
option  (88.2  versus  90.2  for  SIDP  eligibles-but 
not  selected),  there  was  little  difference  in 
MCAT  scores  for  the  different  SIDP  groups  or 
for  those  selected  as  supervisors.  Supervisors 
selected  in  the  terminal  option  had  slightly  lower 
average  ABSR  scores  than  eligibles.  However 
within  the  en  route  option,  those  selected  had 
slightly  high^  ABSR  scores  (41.7  versus  40.6) 

The  largest  difference  between  the  eligible 
and  selected  supervisor  groups  was  evident  in  the 
OKT  scores.  Of  those  initially  entering  the  en 
route  option,  the  average  scores  for  supervisors 
(M  =  33.3)  was  statistically  below  (F(l,466)  = 
7.4,  p<  .01)  that  of  the  ATCSs  who  completed 
Academy  training  and  entered  the  same  option 
but  were  not  yet  selected  (M  =  38.1).  In  con¬ 
trast,  within  those  entering  the  terminal  option, 
eligibles  who  were  selected  exhibited  higher 
average  OKT  scores  (M  =  43.6)  than  those  who 
graduated  from  the  Academy,  af>plied  to  become 
a  supervisor,  or  were  plac^  on  the  list  of  eli- 
gibles  (M  =  39.3,  F  (1,459)  =  7.6,  p<.01). 
The  present  analysis  suggests  that  individuals 
who  enter  the  terminal  option  with  a  more  exten¬ 
sive  background  knowledge  of  ATC  rules  and 
procedures  are  more  likely  to  be  selected  as 
supervisors,  while  the  opposite  proved  to  be  true 
in  the  en  route  environment.  However,  there  are 
many  other  factors  that  may  have  entered  into  the 
selection  process  that  were  not  examined  in  this 
study. 

While  4  of  the  correlations  between  the  selec¬ 
tion  and  experimental  test  scores  and  the  various 
SIDP  m«Bures  for  ATCSs  in  the  en  route  option 
were  statistically  significant,  they  were  all  quite 
low  (see  Table  2).  Of  the  possible  comparisons, 
significant  corrdations  were  noted  between  the 
MCAT  and  TMC  with  the  TECH  peer-supervisor 
ratings  (.07  and  .06)  and  between  ABSR  and  SBI 
TOTAL  (.  10).  Of  the  experimental  tests,  only  die 
DRT  exhibited  a  significant  correlation  with  any 
of  the  SIDP  measures;  a  correlation  of  .16  with 
TECH.  Thus,  there  is  weak  evidence  that  some 
aspects  of  cognitive  ability  at  the  time  of  entry 
into  the  ATCS  occupation  are  predictive  of 
subsequent  technical  ratings  and  likelihood  of 
being  selected  as  a  supervisor. 


As  was  true  for  the  en  route  option,  correla¬ 
tions  between  iqititude  scores  and  SIDP  measures 
for  terminal  option  applicants  were  low  (see 
Table  3).  Ratings  of  communication  skills  were 
positively  correlated  with  ABSR  (r  =  .06, 
p<.05)  and  DRT  (r  =.22,  p<.05)  but  nega¬ 
tively  correlated  with  HT  (r  =  -.17, /?<  .05).  A 
higher  rating  on  interpersonal  skills  was  as¬ 
sociated  with  a  lower  score  on  the  OKT  (r  = 
.06,  p<  .04)  but  higher  ratings  on  ABSR  (r  = 
.08,  p<  .05)  and  DRT  (r  =  .18,  p<  .05).  Tech¬ 
nical  competence  was  significantly  related  to 
DRT  scores  only  (r  =  .22,  p<  .05). 

Results  of  these  analyses  reveal  only  limited 
evidence  of  any  relationship  b^een  any  of  the 
aptitude  measures  and  either  the  peer-super¬ 
visory  ratings  or  the  overall  SBI  rating.  How¬ 
ever,  of  all  the  possible  correlations,  the  MCAT 
and  DRT  were  most  closely  correlated  with  peer- 
supervisory  ratings  2  of  the  technical  (TECH) 
competence  of  the  SIDP  applicants  in  comparison 
to  the  other  ratings.  This  finding  was  consistent 
across  both  options.  The  Abstract  Reasoning  Test 
(ABSR),  which  measures  a  related  yet  different 
aspect  of  the  cognitive  ability  of  ATCSs,  tended 
to  be  more  closely  conelated  with  some  of  the 
other  ratings,  including  the  overall  SBI  rating 
for  en  route  ATCSs. 

DISCUSSION 

These  results  offer  weak  support  for  the  con¬ 
ceit  that  measures  of  aptitudes  for  the  ATCS  job 
taken  prior  to  entry  into  ATCS  training  are 
predictive  of  on-the-job  ratings  of  technical 
proficiency.  These  findings  are  of  increased 
interest  when  one  considers  the  amount  of 
time,  training,  and  other  activities  that  have 
transpired  between  initial  selection  and  subse¬ 
quent  application  to  become  a  supervisor.  Addi¬ 
tionally,  as  trainees  compile  the  various  stages 
from  selection  through  attainment  of  the  foil 
performance  level  status,  continued  loss  of 
unsuccessful  trainees  ensures  that  the  remaining 
workforce  is  relatively  homogenous  with  respect 
to  technical  competence.  Perceptions  of  on-the- 
job  performance  are  therefore  more  likely  due  to 
the  presence  of  untapped  aptitudes  or  diff^erences 
associated  with  various  personality  dimensions 
(e.g.,  self  confidence,  assertiveness).  In  any 
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event,  one  should  be  aware  that  the  aptitude 
measures  included  in  this  study  represent  only  a 
limited  number  of  cognitive  abilities  that  are 
critical  for  on-the-job  performance  of  ATCSs.  It 
could  be  that  a  broader  range  of  aptitude  mea¬ 
sures  would  reveal  more  consistent  and  higher 
correlations  with  the  various  ratings  and  mea¬ 
sures  comprising  the  SIDP. 

Given  the  current  measures  of  aptitudes  for 
the  ATC  profession,  there  was  little  support  for 
any  relationship  between  these  aptitudes  and 
either  overall  ratings  during  the  Skill  Based 
Interview  or  selection  as  a  supervisor.  However, 
background  knowledge  of  ATC  and  pilot  pro¬ 
cedures  may  enhance  the  chance  of  an  indi¬ 
vidual’s  selection  as  a  supervisor  for  those 
entering  the  terminal  option.  One  possible  ex¬ 
planation  is  that  within  the  terminal  option,  the 
ATCSs  need  to  be  more  familiar  with  aircraft 
characteristics  and  associated  rules  and  pro¬ 
cedures  due  to  the  nature  of  the  work  activities. 

Therefore,  those  who  enter  the  terminal 
option  with  a  more  extensive  experience  in  the 
area  and  general  background  knowledge  are  per¬ 
ceived  to  be  more  suited  for  a  supervisory  pos¬ 
ition.  However,  given  the  limit^  number  of 
variables  included  in  this  study  and  the  large 
number  of  additional  factors  that  could  play  an 
explanatory  role,  additional  information  and 
analyses  are  necessary  to  fully  understand  this 
difference. 
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TABLE  1 

Descriptive  statistics  for  the  0PM  seleaion  battery 


MCAT  ABSR  QKT 

M  SD  M  SD  M  SD  N  Group 


Applicants 


69.7 

16.1 

30.5 

9.5 

28.8 

11.3  127,807 

1981-1985' 

En  Route 

90.0 

7.1 

39.8 

5.9 

36.2 

12.9 

5,993 

Academy  entrants' 

91.6 

6.5 

40.5 

5.8 

37.6 

13.2 

3,337 

Passed  Academy 

91.4 

6.5 

40.5 

5.7 

37.6 

12.8 

1,215 

SIDP  iq)plicants 

90.8 

6.5 

40.7 

5.5 

37.5 

12.8 

468 

AH  SIDP  eligibles 

90.8 

6.6 

40.6 

5.6 

38.1 

12.9 

407 

SIDP  eligible  -  not 

91.1 

6.1 

41.7 

4.9 

33.3 

11.4 

61 

Selected 

Terminal  Option 

88.5 

7.3 

39.1 

6.3 

39.4 

13.4 

3,095 

Academy  entrants' 

89.4 

6.9 

39.6 

6.1 

41.2 

13.7 

2,186 

Passed  Academy 

89.5 

6.9 

39.4 

6.3 

40.7 

13.4 

1,084 

SIDP  applicants 

89.8 

6.7 

39.4 

6.2 

40.1 

13.1 

461 

SIDP  eligibles 

90.2 

6.5 

39.6 

6.1 

39.3 

12.9 

85 

SIDP  eligible  -  not 

88.2 

7.3 

38.7 

6.6 

43.6 

13.8 

85 

Selected 

yet  selected 


yet  selected 


NOTES:  'From  Manning,  Della  Rocco,  and  Bryant  (1989) 
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TABLE  2 

Correlations  between  the  0PM  seleaion  tests,  experimental  tests 
and  selected  SIDP  ratings  for  applicants  entering  the  en  route  option 


0PM  &  EXPERI¬ 
MENTAL  TESTS 

Peer-Supervisory  Assessment  Ratings 

COMP 

COMM 

INTR 

LEAD 

TECH 

SBI  TOTAL 

MCAT 

.05 

.03 

.03 

.04 

.07* 

-.05 

TMC 

.04 

.03 

.02 

.03 

.06* 

-.01 

ABSR 

-.01 

.01 

-.01 

-.01 

-.01 

.10** 

OKT 

-.00 

-.02 

-.03 

.00 

.04 

.00 

DHT 

.02 

.04 

-.00 

.02 

.04 

.12 

DRT 

.06 

.04 

.02 

.02 

.16* 

-.08 

*P  <.  05  **p  ^  .01 


TABLES 

Correlations  between  the  0PM  selection  tests,  experimental  tests  and 
seleaed  SIDP  ratings  for  applicants  entering  the  terminal  option 


Peer-Supervisory  Assessment  Ratings 


0PM  &  EXPERI- 


MENTAL  TESTS 

COMP 

COMM 

INTR 

LEAD 

TECH 

SBI  TOTAL 

MCAT 

.03 

.03 

.00 

.03 

.05 

.04 

TMC 

.05 

.05 

.03 

.04 

.05 

.05 

ABSR 

.05 

.06* 

.08* 

.05 

.01 

.04 

OKT 

-.03 

-.02 

-.06* 

-.01 

-.01 

-.02 

DHT 

-.16 

-.17 

-.15 

-.13 

-.14 

.15 

DRT 

.21* 

.22** 

.18* 

.16 

.22** 

.11 

*p  ^  .05  **p  j<  .01 
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MULTIPLEX  CONTROLLER  APTITUDE  TEST  (MCAT) 
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FIGURE  2 


DIAL  READING  TEST 
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2.  AMPERES  16.0  -15.6  14.0  -10.0  -2.0 

3.  ALTITUDE  157.5  15.5  155.0  1.5  152.0 
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AIR  TRAFFIC  CONTROL  SPECIALIST  TECHNICAL  COMPETENCE 
IN  INITIAL  TRAINING  AND  SELECTION  AS  A  HRST-LINE  SUPERVISOR 

Pamela  S.  Della  Rocco,  M.A.  and  Dana  Broach,  Hi.D. 


A  first-line  supervisor  holds  a  key  spot  in  an 
organization,  serving  as  the  main  point  of  contact 
between  the  rank-and-file  and  management  and  as 
the  person  responsible  for  the  day-to-day  or¬ 
ganizational  performance.  The  selection  of  first- 
line  supervisors  was  considered  essen-tially 
settled  as  a  result  of  many  studies  between  1947 
and  1965  (Dooher  &  Marting,  1957;  Raza, 
1987).  Corporations  generally  selected  first-line 
supervisors  on  the  basis  of  their  technical  skills, 
rather  dian  managerial  skills  (Northrup,  Cowin, 
Vanden  Plas,  Fulmer,  Bolick,  Bellace,  &  Rosen- 
zweig,  1978;  Patton,  1974). 

However,  evidence  from  indirect  sources 
suggests  that  such  selection  procedures  for  first 
line  supervisors  may  be  suboptimal.  For  ex¬ 
ample,  Raza  (1987)  noted  that  surveys  of  first 
line  supervisors  suggested  a  need  for  training  in 
communication  and  management  skills.  Myers 
(1990)  reached  much  the  same  conclusions 
through  an  analysis  of  the  FAA  biennial  Job 
Satisfaction  Survey  and  Survey-Feedback-Action 
Program.  Phillips  (1985)  argued  that  while 
technical  skills  and  experience  may  be  important, 
they  are  not  necessarily  rdated  to  the  skills  and 
abilities  required  of  a  successful  supervisor. 
Given  these  two  disparate  lines  of  research,  we 
asked  what  relationship  prior  technical  perfor¬ 
mance  in  training,  for  example,  had  to  promot- 
ability  and  ultimate  selection  for  supervisory 
positk>ns.  To  answer  that  question,  we  inves¬ 
tigated  the  relationship  of  performance  in  initial, 
technical  air  traffic  control  specialist  (ATCS) 
training  to  selection  as  a  first-line  air  traffic 
control  supovisor  through  the  Federal  Aviation 
Administration  Supervisory  Identification  and 
Development  Program  (SIDP). 

Air  Traffic  Control  Specialist  Initial 
Qualification  Training 

Since  1976,  candidates  selected  for  the 
occupation  of  Air  Traffic  Control  Specialist 
(ATCS)  with  the  FAA,  in  the  En  Route  and 
Terminal  option.*!,  were  required  to  complete  a  2 


stage  selection  process.  The  first  stage  consisted 
of  a  battery  of  written  tests  and  was  administered 
by  the  U.S.  Office  of  Personnel  Management 
(0PM,  formerly  the  Civil  Service  Commission) 
prior  to  hiring  by  the  FAA.  The  second  stage  of 
the  ATCS  selection  process  was  a  performance- 
based  pass/fail  initial  training  course  administered 
at  the  FAA  Academy  after  FAA  hire.  This  study 
focuses  on  ATCS  performance  in  the  Academy 
initial  qualification  courses  as  a  potential  predic¬ 
tor  of  selection  as  a  first  line  supervisor. 

ATCSs  work  in  1  of  3  options:  En  Route, 
Terminal,  and  Flight  Service  Station.  En  Route 
and  Terminal  controllers  ensure  the  s^aration 
of  aircraft  by  using  information  about  the  speed, 
direction,  and  altitude  of  aircraft  to  formulate 
clearances  and  communicate  them  to  pilots. 
Clearances  are  srfs  of  instructions  for  pilots, 
designed  to  ensure  the  safe,  orderly,  and  ex¬ 
peditious  flow  of  air  traffic.  En  Route  controllers 
ensure  the  sqiaration  of  aircraft  traveling  be¬ 
tween  airports.  There  are  2  types  of  terminal 
controllers.  Terminal  radar  ^proach  and  d^ar- 
ture  controllers  use  radar  to  s^arate  aircraft 
converging  on  or  dqiarting  from  an  airport. 
Tower  cab  controllers  control  traffic  landing  or 
taking  off  from  the  airport,  most  often  without 
the  direct  use  of  radar.  First-line  En  Route  and 
Terminal  supervisors  oversee  and  coordinate  the 
actions  of  controller  teams  responsible  for  spec¬ 
ific  sectors,  or  segments  of  contiguous  airspace. 
Flight  Service  Station  specialists,  on  the  other 
hand,  provide  other  services  to  pilots,  such  as 
weather  briefings,  filing  flight  plans,  and  helping 
to  locate  lost  aircraft.  Flight  Service  Station 
specialists  do  not  issue  control  instructions  to 
aircraft;  rather,  they  relay  En  Route  or  Terminal 
controller  clearances,  as  appropriate.  Given  the 
fundamental  difference  in  the  options,  we  focused 
on  En  Route  and  Terminal  ATCSs  and  dieir 
supervisors,  as  they  are  responsible  for  issuing 
clearances  to  aircraft,  and  excluded  Flight  Ser¬ 
vice  Station  ATCSs  from  this  analysis. 

The  FA.A  Academy  initial  qualification 
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courses  for  the  En  Route  and  Terminal  options 
were  essentially  based  upon  a  miniaturized 
training  testing  personnel  selection  model  (Siegel, 
1978,  1983;  Siegel  &  Bergman,  1975)  in  which 
individuals  with  no  prior  knowledge  of  the 
occupation  could  be  assessed  for  their  potential  to 
succ^  in  air  traffic  control.  Individuals  who 
failed  were  separated  from  the  occupation. 
Separate  En  Route  and  Terminal  A(^emy 
programs  were  administered  until  October  1985. 
All  candidates  included  in  the  present  study  were 
graduates  of  these  dual  programs.  Each  of  the 
screening  courses,  the  En  Route  Initial  Qualifica¬ 
tion  Training  and  the  Terminal  Initial  Qualifica¬ 
tion  Training,  trained  and  assessed  students  on 
the  application  of  nonradar  procedures  in  each 
type  of  airspace,  respectively.  Although  there 
were  a  number  of  specific  diff^erences  between 
the  2  programs,  both  generally  consisted  of  an 
academic  and  a  laboratory  phase.  Didactic  class¬ 
room  training  on  nonradar  air  traffic  control 
(ATC)  rules  and  principles  was  given  in  the 
academic  phases  of  the  optionspecific  courses. 
Academic  content  included  aircraft  separation 
rules,  cooperative  agreements,  phraseology,  and 
flight  progress  strip-marking.  Students  then 
applied  these  rules,  principles,  and  procedures  of 
ATC  in  a  series  of  thirty-minute  scripted  scen¬ 
arios  in  the  laboratory  phases  of  the  dual  courses 
of  instruction.  Students  were  evaluated  in  acade¬ 
mics  based  upon  multiple  choice  examinations 
and  a  map  test  of  the  synthetic  airspace.  In  the 
laboratory,  students  were  graded  for  their  perfor¬ 
mance  on  6  scripted  scenarios.  A  final  timed, 
paper-and-pencil  exam  assessed  application  of 
ATC  rules.  A  composite  of  these  scores  deter¬ 
mined  pass/fail  status.  The  composite  score  was 
heavily  weighted  on  laboratory  performance.  A 
successful  candidate  was  required  to  achieve  a 
final  composite  score  of  70  or  better. 

Performance  in  the  ATCS  screening  courses 
appears  to  predict  future  performance  in  field 
training  reasonably  well.  For  example.  Manning, 
Della  Rocco,  &  Bryant  (1989)  rqiorted  a  raw 
correlation  of  -.24  (p  ^  .01,  N  =  2,992)  be¬ 
tween  the  En  Route  Initial  Qualification  Course 
final  composite  scores  for  1981-1985  Academy 
graduates  and  status  in  En  Route  field  training. 
Field  training  status  in  that  study  was  coded  1  = 
Full  performance  level,  2  =  In  training  in  or¬ 


iginal  option,  3  =  Switched  options,  and  4  = 
Failed.  Correcting  for  range  restriction  due  to 
explicit  selection  on  the  predictor  resulted  in  a 
corrected  correlation  of  -.44  between  En  Route 
Initial  Qualification  Course  scores  and  field 
training  status.  Manning  et  al.,  also  examined  the 
relationship  of  the  Terminal  Initial  Qualification 
Course  composite  final  scores  to  Terminal  train¬ 
ing  performance.  They  divided  Terminal  con¬ 
trollers  into  2  groups  for  that  analysis:  those 
assigned  to  Terminal  facilities  without  radar 
(visual  flight  rules,  or  "VFR";  N  =  441);  and 
those  assigned  to  Terminal  facilities  with  radar 
("Radar";  N  =  966).  While  the  raw  correlations 
between  Terminal  course  scores  and  status  in 
VFR  training  was  non-significant  (r  =  -.08,  p  > 
.01),  the  initial  qualification  course  score  was 
significantly  relat^  to  various  indices  of  time  in 
training  at  specific  positions  within  the  tower 
cab.  In  contrast,  status  in  the  Radar  Terminal 
training  program  was  predicted  reasonably  well 
by  the  Terminal  Initial  Qualification  Course  final 
composite  scores  (r  =  -.28,  p  ^  .01,  ^  =  962). 
Adjusting  for  restriction  in  range  due  to  explicit 
selection  on  the  predictor  improved  that  overall 
correlation  to  -.51  between  Terminal  course 
scores  and  Radar  Terminal  training  status.  Over¬ 
all,  the  FAA  Academy  ATCS  option-specific 
screening  programs  appeared  to  predict  field 
training  status  reasonably  well.  In  other  words, 
developmental  controllers  who  performed  well 
in  the  FAA  Academy  programs  speared  to  go 
on  to  perform  well  in  technical  field  training. 
While  supporting  data  are  not  easily  available,  it 
may  be  reasonably  supposed  that  persons  who 
did  well  in  the  ATCS  initial  qualification  courses 
and  field  training  were  also  likely  to  do  well 
upon  reaching  full  performance  level. 

This  sophisticated  and  rigorous  screening  and 
on  the  Job  training  system  assured  the  tech-nical 
competence  of  the  ATCS  journeyman  or  full 
performance  level  (FPL)  workforce.  Systematic 
selection  of  first  line  supervisors  of  controllers  is 
accomplished  through  the  SIDP.  These  super¬ 
visors,  as  previously  noted,  oversee  controller 
teams  responsible  for  air  traffic  operating  within 
contiguous  areas  or  sectors  of  airspace.  For 
example,  first-line  En  Route  ATCS  supervisors, 
or  Area  Supervisors,  carefully  monitor  current 
and  anticipated  activities  in  the  assigned  sectors 
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making  up  the  "area  of  specialization."  These 
Area  Supervisors  ensure  that  available  controller 
staffing  is  optimally  deployed  (Federal  Aviation 
Administration,  1989).  Technical  competence  is 
required  of  these  Area  Supervisors  to  interpret 
sector  and  area  activity,  and  to  render  assistance 
to  the  controller  teams  as  needed.  An  analogous 
situation  can  be  found  among  the  supervisors  on 
the  tower  lab  and  Terminal  Radar  Approach 
Control  (TRACON)  facilities.  These  supervisors 
are  selected  from  the  controller  workforce  thr¬ 
ough  the  SIDP. 

The  Supervisory  Identification 
and  Development  Program 

The  SIDP  was  designed  to  emphasize  fac¬ 
tors  such  as  conununication  and  le^ership,  in 
addition  to  technical  capability,  as  supervisory 
selection  criteria.  As  described  by  Myers  (1992), 
SIDP  utilized  a  2-stage  proc^ure  to  select 
supervisors.  Applicants  were  first  rated  by  peers 
and  supervisors  on  a  number  of  dimensions  in 
the  Peer/Supervisory  Assessment  (PSA)  stage  of 
SIDP.  Candidates  scoring  above  the  cutoff  on  a 
composite  of  PSA  ratings  then  participated  in  a 
Skill-Based  Interview  (SBI).  Candidates  who 
were  successful  in  the  SBI  were  then  placed  on 
an  "Eligible  for  Furthor  Consideration"  (EFC) 
list  from  which  supervisors  were  actually  selected 
as  openings  became  available. 

Because  it  may  be  inqwrtant  that  supervi¬ 
sors  in  the  ATCS  occupation  demonstrate  tech¬ 
nical  competence,  we  hypothesized  that  success¬ 
ful  SIDP  candidates,  in  general,  would  demon¬ 
strate  better  technical  performance  as  early  in 
their  careers  as  the  FAA  Academy  initial  qualifi¬ 
cation  courses.  We  reasoned  that  (a)  if  persons 
doing  wdl  in  initial  training  did  well  in  field 
training,  as  the  available  ditta  suggested,  and  that 

(b)  if  persons  doing  well  as  FPL  controllers,  then 

(c)  such  persons  were  more  likely  to  be  selected 
into  first-line  supervisory  positions  through 
SIDP.  We  tested  this  reasoning  by  investigating 
the  following  3  researdi  questions: 

(1)  How  did  scores  in  the  initial  qualification 
training  courses  correlate  with  subsequent 
SIDP  PSA  measures  that  included  a  rating 
of  technical  competence? 


(2)  Did  SIDP  iqiplicants,  selected  through  the 
SIDP  program,  demonstrate  better  technical 
skills  in  the  initial  qualification  courses  than 
unsuccessful  SIDP  iqiplicants? 

(3)  Did  Academy  initial  qualification  course 
scores  predict  future  selection  as  a  first-level 
supervisor  in  the  SIDP  program? 

METHOD 

Subjeas 

ATCSs  who  entered  the  FAA  Academy  in 
either  the  En  Route  or  Terminal  screening  pro¬ 
grams  between  August  1981  and  S^tember  1985 
and  applied  to  the  SIDP  between  1985  and  1988 
were  selected  for  these  studies.  Some  applicants 
changed  options  after  they  passed  the  FAA 
Academy  initial  qualifications  programs.  For 
purposes  of  this  analysis,  only  those  applicants 
who  completed  the  Academy  and  were  still 
working  in  their  original  option  w^e  included. 
Of  the  1,128  applicants  meeting  these  sampling 
requirements,  ^8  were  in  the  En  Route  option, 
while  660  were  in  the  Terminal  option. 

All  of  the  controllers  in  this  sample  entered 
on  duty  with  the  FAA  after  the  illegal  1981 
strike  and  before  the  FAA  Academy  program 
changed  in  October,  1985.  The  sample  was 
predominantly  male  (85%)  and  nonminority 
(93%).  The  modal  formal  education  complied 
was  "some  college"  (48%),  with  about  a  third 
(33%)  having  completed  a  baccalaureate  degree. 
About  a  third  (34%)  had  at  least  some  prior 
aviation-related  experience  prior  to  joining  die 
agency. 

Three  groups  of  ^plicants  were  compared, 
based  upon  success  in  the  SIDP  selection  pro¬ 
cess:  (1)  candidates  who  were  successful  on  the 
PSA  and  referred  to  the  SBI  {Referred  versus 
Not  r^erred);  (2)  based  upon  the  SBI,  candi¬ 
dates  were  deemed  "eligible  for  consideration" 
(EFC)  or  not  (EFC  versus  Not  EFQ;  and  (3) 
candidates  selected  as  supervisors  from  the  EFC 
list  or  not  {Selected  versus  Not  selected). 

Three-quarters  of  the  sample  {N  =  844)  were 
referred  to  the  Skill-Based  Interview  stqi  in  the 
SIDP.  About  half  (55%,  N  =  615)  of  the  total 
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saiiq)le,  or  three-quartm  of  those  referred  to  the 
SBI,  were  rated  as  "Eligible  for  Further  Con¬ 
sideration"  in  the  SIDP.  Just  14%  (N  =  160)  of 
the  total  sample,  or  26%  of  the  persons  rated  as 
"Eligible  for  Further  Consideration,"  were 
selected  as  first-line  supervisors  as  of  March 
1991.  T^le  1  presmts  data  on  ^plicant  status  in 
SIDP  by  career  option. 

Measures 

Thirteen  performance  measures  were  collec¬ 
ted  on  eadi  student  during  approximately  9 
wedcs  at  the  FAA  Academy.  Five  of  the  meas¬ 
ures  were  multiple-choice  tests  covering  the 
academic  materials.  These  assessed  the  candi¬ 
date’s  ability  to  learn  and  retain  the  basic  know¬ 
ledge  required  for  the  job.  A  sixth  measure  was 
a  map  test,  which  assessed  the  student’s  ability  to 
learn  a  map  of  the  relatively  simple  synthetic 
airspace.  These  6  measures  account^  for  10%  of 
the  student’s  final  score.  For  purposes  of  this 
piq>er,  scores  from  4  multiple-choice  tests  and  the 
map  test  wwe  combined  into  a  Block  Test  Aver¬ 
age  (BA).  The  fifth  multiple  choice  test  was  a 
comprdiensive  final  exam  on  the  academic 
phase,  the  Comprehensive  Phase  Test  (CPT),  and 
was  analyzed  as  a  separate  measure. 

In  the  laboratory  phase,  six  30-minute  stan¬ 
dardized  laboratory  problems  were  formally 
graded.  A  student  received  two  scores  from  a 
grading  instructor  on  each  problem.  The  first 
score,  the  technical  assessment  (TA),  was  a 
numeric  assessment  of  the  student’s  errors  in 
applicatmn  of  ATC  rules  and  procedures.  The 
second  score,  the  instructor  assessment  (lA),  was 
a  subjeaive  instructor  rating  of  the  student’s 
global  performance  on  the  problem.  These  2 
scores  were  averaged  for  the  final  problem  score. 
The  student’s  lowest  score  of  the  6  laboratory 
problems  was  dropped.  Thus,  only  5  of  the  6 
graded  problems  were  counted  toward  the  final 
comprdiensive  course  score.  The  graded  labor¬ 
atory  simulation  problems  comprised  65%  of  the 
final  score.  Finally,  the  students  were  given  a 
timed,  multiple-choice  test,  the  Controller  Skills 
Test  (CST),  which  was  a  pq>er-and-pencil  assess¬ 
ment  of  the  student’s  ability  to  apply  ATC  rules 
and  procedures.  This  test  comprised  25%  of  the 
Final  Comprdiensive  Score  (COMP).  Table  2 


summarizes  these  FAA  Academy  initial  qualifi¬ 
cations  training  performance  measures. 

Analyses 

Because  the  En  Route  and  Terminal  Initial 
Qualification  Courses  were  diffident  programs 
utilizing  measures  similar  in  name  only,  analyses 
wo-e  conducted  for  the  En  Route  and  tnminal 
options  separately.  Analyses  were  conducted  to 
address  the  following  research  questions: 

Research  Question  1 

What  was  die  relationship  b^een  FAA 
Academy  technical  performance  and  PSA  ratings, 
especially  the  technical  comp^ence  PSA  rating? 
We  hypothesized  that  there  would  be  a  non-zero 
correlation  b^een  the  FAA  Academy  measure 
of  technical  performance  and  PSA  ratings,  par¬ 
ticularly  the  PSA  technical  competence  category 
or  dimension. 

Research  Question  2 

Was  performance  in  FAA  Academy  initial 
technical  qualification  training  better  for  suc¬ 
cessful  than  for  unsuccessful  SIDP  ^plicants  at 
each  stage  of  SIDP?  We  hypothesized  that  there 
would  be  significant  differences  in  FAA  Acad¬ 
emy  scores  for  successful  compared  to  unsuc¬ 
cessful  participants  at  each  stage  of  SIDP.  We 
used  multiple  analysis  of  variance  (MANOVA)  to 
test  diis  hypothesis. 

Research  (Question  3 

Did  performance  in  the  FAA  Academy  ini¬ 
tial  ATCS  qualification  training  program  predict 
success  at  each  stage  of  SIDP?  We  hypothesized 
that  higher  performance  in  the  FAA  Academy 
would  improve  the  odds  for  success  at  each  stage 
of  SIDP.  We  used  logistic  regression  (Hosmer  & 
Lemeshow,  1989;  Norusis,  1990)  to  test  this 
hypothesis. 

RESULTS 

Research  question  1:  Academy  performance  - 
PSA  ratings 
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Because  the  PSA  contained  an  evaluation  of 
the  ^licant’s  technical  abilities,  an  analysis  was 
conducted  to  determine  the  extent  of  the  relation¬ 
ship  between  Academy  measures  and  PSA  rat¬ 
ings.  Table  3  presents  the  correlations  among  the 
SIDP  PSA  measures  and  Academy  scores  for  the 
En  Route  and  Terminal  options.  Results  of  these 
analyses  revealed  that  there  were  small  but 
statistically  significant  correlations  between  all  of 
the  Academy  measures  and  the  PSA  Technical 
rating,  except  for  the  CPT,  for  the  En  Route 
option.  In  Ae  Terminal  option,  however,  the 
hypothesized  correlations  between  the  PSA 
Technical  ratings  and  laboratory  measures  were 
not  found. 

Research  question  2:  Academy  performance  - 
SIDP  outcomes 

The  purpose  of  research  question  2  was  to 
compare  Academy  performance  of  successful 
SIDP  applicants  to  unsuccessful  applicants  it 
each  stage  of  the  SIDP  selection  process.  We 
hypothesized  that  successful  SIDP  applicants 
demonstrated  better  technical  performance  as 
early  in  their  career;  as  the  Academy  initial 
qualification  training  courses.  Table  4  presents 
descriptive  statistics  for  the  SIDP  ^plicants’ 
Academy  performance  scores  by  outcome  at  each 
stage  of  SIDP.  Results  of  MANOVAs  utilizing 
the  Academy  variables  as  dq)endent  variables 
revealed  no  statistically  significant  differences 
between  groups  for  (1)  referral  to  SBI  for  either 
En  Route  or  Terminal,  (2)  eligibility  for  con¬ 
sideration  for  Terminal  only,  and  (3)  selection 
for  either  En  Route  or  Terminal  supervisor.  A 
MANOVA  comparing  the  En  Route  EFC  and  Not 
EFC  groups  was  significant  (F  =  2.73,  p  <_ 
.01).  Univariate  analyses  of  variance  {df  =  1, 
349  in  each  analysis)  demonstrated  significant 
differences  b^een  EFC  and  Not  EFC  groups  on 
the  following  Academy  variables:  AVLS  (F  = 
7.33,  p  ^  .01);  AVIA  (F  =  12.96,  p  <,  .001); 
AVTA  (F  =  5.61,  p  <  .05);  and  COMP  (F  = 
4.83,  p  ^  .05).  The  mean  score  for  each  of 
these  variables  was  lower  for  the  EFC  group  than 
for  the  Afor  EFC  group.  The  hypothesis  that 
successful  SIDP  candidates  had  demonstrated 
better  Academy  performance  than  their  unsuc¬ 
cessful  SIDP  counterparts  could  not  be  sup¬ 
ported. 


Research  question  3:  Academy  performance 
and  the  odds  for  success  in  SIDP 

As  ATCS  initial  qualification  course  scores 
predicted  technical  job  performance,  and  since 
that  technical  competence  was  required  of  con¬ 
troller  first-line  supervisors,  we  hypothesized  that 
better  performance  in  the  ATCS  Academy  initial 
qualification  courses  improved  the  odds  for 
referral  to  the  Skill-Based  Interview,  for  making 
the  Eligible  for  Consideration  list,  and  for  selec¬ 
tion  as  a  supervisor.  The  independent  variables, 
rq)resenting  Academy  performance,  were:  acade¬ 
mics  block  test  average  (BA);  academics  com¬ 
prehensive  phase  test  (CPT);  average  of  the  6 
graded  laboratory  problem  technical  performance 
assessments  (A>^A);  average  of  the  6  graded 
laboratory  problem  instructor  assessments  of 
potential  (AVIA);  and  score  on  the  final  Con¬ 
troller  Skills  Test  (CST).  These  independent 
variables  were  logistically  regressed  on  the 
categorical  outcomes  at  each  stage  of  SIDP. 
Results  of  the  logistic  regression,  used  to  test  this 
hypothesis,  are  presented  below. 

Referral  to  SBI.  The  probability  of  being 
referr^  to  the  SBI  was  estimated  using  the  SPSS 
logistic  regression  procedure  (Norusis,  1990);  the 
independent  variables  were  entered  into  the 
equation,  and  model  goodness-of-fit  evaluated. 
The  analysis  indicated  that  previous  performance 
in  technical  training  did  not  predict  referral  to 
SBI  in  either  the  En  Route  or  Terminal  options. 
Assessment  of  the  goodness-of-fit  using  -2  times 
the  log  likelihood  (-2  LL)  of  the  model  indicated 
a  poor  fit  to  the  data  (En  Route:  -2  LL 
X*(467)=526.3,  p  <  .05;  Terminal:  -2  LL 
X*(658)  =  737.0,  p  <  .01),  where  the  null 
hypothesis  was  that  the  model  fit  the  data. 
Inspection  of  the  classification  table  resulting 
from  the  model  supported  the  conclusion  of  a 
poor  model  fit.  \^ile  the  overall  correct 
classification  rate  of  75.0%  and  74.7%,  for  En 
Route  and  Terminal,  respectively,  appeared 
accq)table,  all  but  2  of  the  Terminal  cases  not 
actu^ly  referred  to  the  SBI  were  predicted  as 
referrals.  In  other  words,  the  model  predicted 
that  all,  except  2,  cases  would  have  been  referred 
to  the  SBI.  Thus,  performance  in  the  ATCS 
initial  qualification  courses  did  not  provide  useful 
information  about  characteristics  predicting 
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referral  to  the  Skill-Based  Interview. 

Eligibility  for  Consideration.  The  probability 
of  making  the  "Eligible  for  Consideration"  list 
after  referral  to  the  SBI  was  estimated  by 
entering  the  ind^>endent  variable  into  the 
equation,  and  evaluating  model  goodness-of-fit. 
The  analysis  indicated  that  previous  technical 
training  performance  did  not  predict  making  the 
EEC  list  from  the  Skill-Based  Interview.  Assess¬ 
ment  of  the  goodness-of-fit  using  the  -2  times  the 
log  likelihood  of  the  model  indicated  a  poor  fit  to 
the  EEC  data  (En  Route:  -2  LL  ;^(348)  = 
399.2,  p  <  .05;  Terminal:  -2  LL  x'(491)  = 

563.4,  p  <  .05)  under  the  null  hypothesis  that 
the  nnodel  did  fit  the  data.  The  overall  rate  of 
correct  classifications  by  the  model  was  72.6% 
for  En  Route  and  73.4%  for  Terminal.  However, 
the  model  predicted  that  all  but  6  En  Route  and 

1  Terminal  <q>plicant  would  have  made  the  EEC 
list.  This  result  indicated  that  performance  in 
initial  technical  training  failed  to  provide  useful 
information  for  the  practical  prediction  of  making 
the  EEC  list  from  the  interview  process. 

Seleaion  as  a  supervisor.  Einally,  the  prob¬ 
ability  of  selection  from  the  EEC  list  as  a  super¬ 
visor  was  estimated  by  once  again  entering  the 
independent  variables  into  the  equation  and 
assessing  model  goodness-of-fit.  This  analysis 
also  indicated  that  previous  performance  in  initial 
training  did  not  predict  actual  selection  for  a 
first-line  supervisory  position  in  the  air  traffic 
controller  occupation.  The  null  goodness-of-fit 
hypothesis  that  the  model  fit  the  data  for  the 
Terminal  option  was  rejected  based  on  -2  times 
the  log  likelihood  of  the  model  (-2  LL  x^361)  = 

428.4,  p  <  .01)  despite  an  overall  rate  of  70.8% 
correct  classifications.  This  time  the  model 
predicted  that  no  cases  would  been  selected  as  a 
supervisor  from  the  Terminal  option  EEC  list. 
Eor  the  En  Route  option,  the  null  goodness-of- 
fit  hypothesis  was  not  rejected  based  upon  the  - 

2  times  the  log  likelihood  of  the  model  (-2LL 
X*(251)  =  261.9,  p  <  .31).  Although  the  overall 
correct  classifications  from  the  model  was 
78.6%,  no  selections  from  the  EEC  list  were 
predicted  based  upon  this  model  using  perfor¬ 
mance  in  initial  technical  training.  As  with  the 
previous  logistic  regression  analyses,  the  Acad¬ 
emy  measures  of  technical  performance  failed  to 


provide  useful  information  for  predicting  selec¬ 
tion  as  a  supervisor. 

DISCUSSION 

The  purpose  of  this  study  was  to  examine  the 
relationship  of  success  in  the  Supervisory  Iden¬ 
tification  and  Development  Program  and  prior 
performance  in  the  EAA  Academy  ATCS  initial 
qualification  courses.  Previous  studies  had  dem¬ 
onstrated  significant  correlations  between  Acad¬ 
emy  scores  and  subsequent  field  training  perfor¬ 
mance  (Manning,  Della  Rocco,  &  Bryant,  1989; 
Della  Rocco,  Manning,  &  Wing,  1990).  The 
basic  premise  that  persons  who  perform  well  in 
initial  technical  training  for  a  technically  demand¬ 
ing  occupation  would  be  more  likely  to  be  sel¬ 
ected  for  first-line  supervisory  positions  in  that 
occupation  was  tested  from  three  perspectives. 

The  first  research  question  examined  the 
relationship  between  Ac^emy  scores  and  the 
ratings  from  the  Peer/Supervisory  Assessment. 
Small,  significant  correlations  were  found  be¬ 
tween  Academy  scores  and  the  PSA  technical 
competence  assessment  for  the  En  Route  option. 
Thus,  there  was  some  evidence  tha  En  Route 
Academy  scores  were  related  to  Pull  Perfor¬ 
mance  Level  (PPL)  or  journeyman-level  perfor¬ 
mance.  The  reason  that  the  correlations  were 
small  is  not  evident  because  greater  correlation 
has  been  r^rted  between  these  Academy  scores 
and  field  training  performance  (Manning,  a  al., 
1989).  However,  candidates  completed  the 
Academy  up  to  7  years  prior  to  the  assessment 
for  SIDP  which  is  a  considerable  length  of  time 
between  the  2  sets  of  assessments.  The  predicted 
relationship  between  Academy  laboratory  mea¬ 
sures  and  PSA  technical  ratings  was  not  found 
for  the  Terminal  option,  however.  This  finding 
corresponds  to  findings  of  Manning  et  al.,  (1989) 
of  the  correlations  between  the  Academy  and 
subsequent  field  training  measures  for  terminal 
option  controllers. 

The  second  research  question  assessed  the 
mean  differences  in  Academy  scores  between 
successful  and  unsuccessful  applicants  to  SIDP 
at  3  stages  of  the  SIDP  selection  process.  The 
hypothesis  that  successful  applicants  would  have 
performed  better  in  the  Academy  courses  was  not 
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supported.  Although  the  average  of  most  of  the 
Ac^emy  scores  were  slightly  higher  for  the 
successful  candidates,  the  difference  was  not 
significant.  In  fact,  the  successful  candidates 
deemed  "eligible  for  further  consideration"  had 
statistically  significantly  lower  average  scores 
than  the  unsuccessful  candidates  in  the  En  Route 
option.  No  significant  differences  were  found 
bkween  any  of  the  SIDP  groups  in  the  Terminal 
q[>tion. 

The  final  research  question  investigated  the 
extent  to  which  Academy  scores  predicted  selec¬ 
tion  in  SIDP  using  logistic  r^ession.  The  an¬ 
alyses,  based  on  a  relatively  large  sample  of 
controllers,  indicated  that  performance  in  initial 
technical  training  was,  in  fact,  not  predictive  of 
referral  to  the  initial  stages  of  a  Supervisory 
Identification  and  Development  Program  nor  of 
selection  as  a  supervisor. 

On  one  hand,  these  data  lend  support  to  the 
contention  by  Phillips  (1985)  diat  technical  skills 
and  experience  may  be  unrelated  to  the  skills  and 
abilities  required  of  sup^isors.  On  the  other 
hand,  it  may  be  argued  that  the  initial  technical 
training  is  so  distant,  both  temporally  and  con¬ 
ceptually,  from  the  supervisory  selection  process 
as  to  1m  meaningless.  Indicators  of  t^nical 
performance  measured  closer  in  time  to  con¬ 
sideration  for  SIDP,  such  as  performance  in  field 
training,  may  be  more  indicative  of  managmal 
potential.  Another  alternative  interpretation  is 
that  the  level  of  homogoious  technical  com¬ 
petence  within  the  workforce  is  assured  by  the 
nature  of  die  upfront  screening  process  repre¬ 
sented  by  die  ATCS  initial  qualification  courses. 
Thus,  ofoer  factors,  such  as  communication 
skills,  knowledge  of  agency  programs,  nuy  be 
more  predictive  of  successfol  completion  of  the 
multipie  hurdles  embedded  within  SIDP. 
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TABLE  1 

Sample  status  at  each  stage  of  SID P  by  ATC  option 


Status 

Option 

En  Route 

Terminal 

Referral  to  Skill-Based  Interview  (SBI) 

Referred 

351 

494 

(75%) 

(75%) 

Not  Referred 

117 

166 

(25%) 

(25%) 

Eligible  for  Further  Consideration 

Eligible 

252 

363 

(72%) 

(73%) 

Not  Eligible 

99 

131 

(28%) 

(27%) 

Selected  for  a  Supervisory  Position 

Selected 

54 

106 

(21%) 

(29%) 

Not  Selected 

198 

257 

(79%) 

(71%) 
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TABLE  2 

FAA  Academy  ATCS  initial  qualifications  training  measures 


I%ase 

Weight 

Measure 

Descriptimi 

Academics 

10% 

Block  test  average  (BA) 

Comprdiensive  Phase  Test 
(CPT) 

4  multiple-choice  instructional  block  tests 
and  map  test 

comprdieiisive  multiple-choice  test  over  all 
academic  topics 

Laboratory 

65% 

Average 

(AVTA) 

Average 

(AVIA) 

Average 

(AVL5) 

technical  score 

instructor  score 

laboratory  score 

average  technical  performance  assessments 
on  5  out  of  6  graded  laboratory 

problems 

average  instructor  assessment  of  potential 
on  b^  5  out  of  6  graded  limoratory 
problems 

average  overall  score  on  best  5  out  of  6 
graded  laboratory  problems 

Final  examination 

25% 

Controller  Skills  Test  (CST) 

con^>reh«isive,  multiple-choice  examiiution 

Final  score 

Final  conqx>site  score  (COMP) 

70%  required  to  pass 

TABLE  3 

Correlations  between  FAA  Academy  performance  measures  and  PSA  ratings  by  ATC  option 


PSA  Rating  Categories 


Measure  Communication  Leadership  Interpersonal  Tedinical  Conqxwite 


En  Route 


BA 

.061 

.092* 

.080 

.104* 

.089 

CPT 

.059 

.085 

.050 

.081 

.073 

AVL5 

.034 

.023 

-.032 

.106* 

.032 

AVTA 

.028 

.020 

-.034 

.097* 

.027 

AVIA 

.046 

.031 

-.013 

.107* 

.042 

CST 

.106* 

.118* 

.089 

.143* 

.120* 

COMP 

.068 

.065 

.006 

.144* 

.072 

Terminal 


BA 

.103* 

.081* 

.066 

.039 

.079* 

CPT 

.151* 

.128* 

.127* 

.084* 

.132* 

AVL5 

.040 

.044 

-.004 

.046 

.033 

AVTA 

.056 

.070 

.016 

.063 

.055 

AVIA 

.007 

.027 

-.030 

.018 

.005 

CST 

.071 

.093* 

.043 

.081* 

.079* 

COMP 

.080* 

.090* 

.028 

.082* 

.075 

*p  <..05 
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TABLE  4 

Comparison  of  FAA  Academy  performance  measures  by  ATC  t^ion  and  outcome  at  each  stage  ofSIDP 
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RELATIONSHIPS  BETWEEN  PERFORMANCE  IN  AIR  TRAEFIC  CONTROL  SPECIALIST 
TECHNICAL  TRAINING  AND  SUPERVISORY  SELECTION  PROGRAMS 

Carol  A.  Manning,  Ph.D. 


After  successfully  completing  the  Federal 
Aviation  Administration’s  (FAA)  2-stage  selec¬ 
tion  procedure,  as  described  in  Della  Rocco  and 
Broach  (1991),  Air  Traffic  Control  Specialists 
(ATCSs)  ent^  technical  training  programs  in 
their  respective  field  facility  assignments.  The 
type  and  duration  of  the  training  d^ends  on  the 
type  of  facility  (option)  and  amount  of  aircraft 
traffic  controlled  by  the  facility.  A  brief 
discussion  of  the  types  of  air  traffic  control 
facilities  will  be  presented,  followed  by  a 
description  of  the  training  programs  provided  by 
each  facility  type. 

ATCS  Options 

Air  Traffic  Controllers  in  the  FAA  can  be 
split  into  3  options  or  specialties:  en  route, 
terminal,  and  flight  service  station  (FSS).  En 
route  and  terminal  ATCSs  ensure  the  s^aration 
of  aircraft  traveling  between  airports  (en  route) 
and  approaching  or  departing  from  airports 
(terminal)  by  formulating  and  issuing  clearances 
(sets  of  instructions  for  aircraft  regarding  their 
impropriate  altitudes  and  directions  of  flight). 
The  clearances  are  designed  to  ensure  aircraft 
sq>aration  and  maximize  fuel  efficiency.  Flight 
Service  Station  specialists  provide  services  to 
pilots  such  as  giving  weather  briefings,  filing 
flight  plans,  and  helping  to  locate  lost  aircraft. 
For  this  study,  the  relationship  between  tedinical 
training  performance  and  performance  in  the 
Supervisory  Identification  and  Development 
Program  (SIDP)  was  limited  to  en  route  and 
terminal  controllers,  because  the  screening, 
training  and  technical  requirements  of  the  FSS 
specialist’s  job  are  considerably  different  than 
those  for  ATCSs  in  the  other  options. 

ATCS  Field  Technical  Training 

In  spite  of  some  similarities  in  job  functions, 
the  technical  training  provided  for  en  route  and 
terminal  controllers  is  very  different,  and  even 
within  different  types  of  terminal  facilities,  the 
training  differs.  Much  of  the  reason  for  the  dif¬ 


ferences  in  training  is  that  air  traffic  control 
procedures  differ  according  to  the  facility  to 
which  the  student  is  assigned.  At  en  route 
facilities,  traffic  is  usually  moving  rapidly  and  at 
high  altitudes  and  must  be  k^t  further  than 
at  terminal  facilities,  where  traffic  is  slowing 
down  and  converging  on  a  single  location.  Thus, 
sqiaration  standards  (the  minimum  distance  that 
aircraft  must  be  k^t  im^)  vary  by  type  of  ATC 
facilities  ami  the  training  varies  as  well. 

In  the  en  route  option,  the  unit  of  air  traffic 
control  operation  is  the  sector,  a  piece  of  air¬ 
space  for  which  a  team  of  2  or  3  controllers  is 
responsible  (during  times  of  slow  traffic,  only  1 
controller  may  be  responsible  for  a  sector).  A 
group  of  between  S  and  8  sectors  is  combined 
into  what  is  called  an  area  of  specialization.  An 
en  route  controller  is  assigned  to  only  1  area  of 
specialization,  but  is  responsible  for  learning  to 
control  traffic  in  all  sectors  within  that  area.  The 
team  of  en  route  controllers  working  at  most 
sectors  handles  duties  related  to  radar  separation 
of  aircraft  (radar  duties,  including  formulating 
clearances  to  ensure  separation  and  delivering 
them  by  radio  to  pilots,  handing  off  responsibility 
for  an  aircraft  to  another  controller),  duties  to 
assist  the  radar  controller  (radar  associate  duties, 
including  maintaining  records  about  clearances 
that  have  been  issued  or  other  changes  in  the 
flight  plan  of  an  aircraft,  identifying  potential 
problems,  communicating  information  not 
directly  related  to  aircraft  separation  of  aircraft  to 
pilots  or  other  controllers),  or  other  support 
activities  (assistant  controller  duties,  including 
entering  data  into  the  computer,  ensuring  that  all 
records  of  flight  progress  are  available  for  the 
controller  in  charge).  En  route  controllers  are 
trained  on  assistant  controller  duties  first,  then 
are  given  training  on  increasingly  difficult  re¬ 
sponsibilities  (radar  associate  duties,  then  radar 
duties).  Training  on  concqits  is  conducted  first 
in  the  classroom,  then  ^plied  in  a  laboratory 
setting,  then  reinforced  during  on-the-job  training 
(OJT)  conducted  in  a  supervised  setting.  At  some 
facilities,  all  radar  associate  training  is  completed 
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on  each  sector  before  radar  training  begins;  at 
othw  facilities,  both  radar  associate  and  radar 
training  are  provided  for  a  specific  sector  before 
training  begins  on  the  next  sector.  En  route 
controllers  must  complete  up  to  9  phases  of  field 
training,  dqiending  on  the  type  of  training 
provided  by  the  facility. 

In  the  terminal  option,  there  are  2  different 
types  of  controllers.  At  approach  control  facil¬ 
ities,  the  terminal  controllers  use  radar  equipment 
to  sq;iarate  aircraft  converging  on  or  d^arting 
from  an  airport,  although  these  aircraft  are 
slower  and  closer  together  than  they  are  in  the  en 
route  environment.  Approach  controllers  may 
work  in  pairs  but  often  work  alone  to  manage  the 
aircraft  within  their  radar  position  (equivalent  to 
an  en  route  sector).  Thus,  training  for  terminal 
approach  controllers  generally  involves  the 
combined  duties  of  a  radar  controller,  instead  of 
splitting  out  the  functions  into  several  training 
phases,  as  is  seen  in  the  en  route  environment. 
Also,  iq)proach  controllers  can  be  required  to 
train  to  journeyman  level  in  up  to  10  sectors 
(called  "positions*  in  the  terminal  environment) 
in  comparison  to  a  maximum  of  8  sectors  for  en 
route  controllers.  Approach  controllers  take  at 
least  2  leases  of  fidd  training  (flight  data  and 
radar,  a^  nonradar  training  is  optional)  but  may 
take  up  to  6  phases  if  their  facility  is  combined 
with  a  tower  cab. 

Tower  cab  controllers  control  traffic  landing 
or  taking  off  from  an  airport.  At  tower  cab 
facilities,  controllers  perform  duties  associated 
with  different  indq)endent  functions,  e.g., 
(entering  and  updating  flight  data,  delivering 
clearances  related  to  the  flight  to  pilots  who  have 
not  yet  entered  the  taxiway,  directing  air  traffic 
moving  along  the  taxiways  back  and  forth  from 
the  runway,  and  directing  aircraft  to  take  off  and 
land).  Some  of  the  functions  may  be  combined, 
but  the  separate  functions  are  not  usually 
performed  as  a  team.  Thus,  training  is  conducted 
on  1  function  at  a  time.  Tower  cab  controllers 
complete  4  phases  of  field  training. 


Available  Measures  of  Field 
Training  Performance 

Several  types  of  information  on  performance 
are  obtained  for  each  phase  of  air  traffic  control 
field  training:  the  start  and  completion  dates,  the 
number  of  hours  used  to  complete  OJT,  and  the 
grade  (Pass,  Fail,  or  Withdraw).  A  rating  of 
control!^  potential,  measured  on  a  1  to  6  scale, 
is  made  by  an  instructor  or  supervisor  who  most 
frequently  observed  the  student  during  that  phase. 
This  information  can  be  compiled  to  derive 
measures  of  training  performance,  such  as  the 
amount  of  time  (in  years)  required  to  reach 
journeyman  or  full  performance  levd  (FPL) 
status,  mean  instructor  ratings  computed  across 
certain  training  phases,  time  (in  days)  to 
complete  OJT  in  certain  training  phases,  a^  total 
number  of  OJT  hours  required  to  complete  those 
phases.  Total  OJT  hours  and  days  graerally  vary 
by  facility  type  (Manning,  Della  Rocco  & 
Bryant,  1989).  For  example,  en  route  controllers, 
average  2.7  years  to  reach  FPL  status,  while 
terminal  approach  controllers  average  1 .7  years; 
average  training  completion  time  for  tower  cab 
controllers  is  .7  years. 

Although  these  assessments  of  training  per¬ 
formance  are  available  for  most  students,  it  must 
be  understood  that  a  number  of  outside  factors 
(besides  iq)titude  and  technical  proficiency)  may 
affect  the  accuracy  of  their  measurement.  For 
example,  time  to  reach  FPL  status  may  be 
affected  by  delays  in  training  caused  by  the  need 
to  use  the  controller  in  an  operational  position, 
the  number  of  other  students  undergoing  OJT  on 
the  same  airspace,  and/or  the  availability  of  the 
training  laboratory.  The  number  of  OJT  hours 
used  may  be  affected  by  insufficient  exposure  to 
different  types  of  traffic  during  training.  The 
subjective  rating  of  student  potential  could  be 
affected  by  a  number  of  rating  biases  familiar  to 
psychologists  (e.g.,  leniency,  central  tendency, 
severity,  halo  effect,  contrast  and  similarity 
errors).  Withdrawal  from  training  before 
completion  usually  occurs  because  of  training 
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failure,  but  students  occasionally  withdraw  from 
the  training  program  of  dieir  own  accord. 

Despite  the  measurement  problems  associated 
with  these  training  performance  measures,  they 
are  die  best  measures  currently  available  to 
describe  performance  in  ATCS  technical  training 
programs.  Some  of  the  measures  described 
above  have  been  used  as  criteria  against  which  to 
validate  ATCS  selection  procedures  (Manning, 
K^,  &  Bryant,  1989).  Statistically  significant 
(though  somewhat  small)  correlations  were  found 
between  the  Office  of  Personnd  Management 
rating  and  training  status,  instructor  ratings,  and 
time  to  reach  FPL  (a  negative  correlation)  for 
students  in  the  en  route  option;  somewhat  higher 
correlations  were  found  b^een  the  Academy 
score  and  die  same  measures  of  field  training 
p^onnance. 

For  students  assigned  to  terminal  radar 
^proach  control  facilities,  the  0PM  score  pre¬ 
dicted  training  status,  instructor  ratings,  and  time 
to  reach  FPL,  while  Academy  scores  only  pre¬ 
dicted  training  status  and  instructor  ratings.  For 
students  assigned  to  tower  cabs,  the  0PM  score 
was  not  predictive  of  any  measure  of  field  train¬ 
ing  performance.  However,  the  Occupational 
Knowledge  Test  score,  whidi  adds  extra  credit 
points  to  the  earned  rating  for  those  who  demon¬ 
strate  job  knowledge,  was  significandy  corrdated 
with  days  and  hours  in  training  phases,  as  well 
as  time  to  reach  FPL  status  and  instructor  rat¬ 
ings.  The  Academy  score  was  significandy 
correlated  with  all  training  times  and  instructor 
ratings  for  students  at  tower  cab  facilities. 

For  this  study,  one  question  of  interest  was 
how  well  the  available  measures  of  field  training 
performance  would  predict  the  ratings  which 
determined  whether  the  candidate  progressed  to 
the  next  stage  of  supervisory  selection.  It  was 
expected  that  measures  reflecting  technical 
proficiency,  such  as  training  status  or  times  to 
complete  certain  portions  of  the  training 
program,  would  predict  ratings  of  technical 
competence  used  in  the  Peo'-Supervisory 
Assessment  (PSA)  phase  of  the  SIDP  but  would 
be  unrelated  to  other  types  of  ratings.  On  the 
other  hand,  instructor  ratings  of  student  potential 
might  be  related  to  ratings  reflecting  other 


qualities  of  the  candidate,  such  as  interpersonal 
skills. 

Another  question  of  interest  was  how  well 
the  available  measures  of  field  training  perfor¬ 
mance  would  predict  success  in  the  3  stages  of 
the  supervisory  selection  process.  It  was  also 
expected  that  instructor  ratings  of  student 
potential  might  be  predictive  of  referral  to  the 
Skill-Based  Interview  (SBI),  which  is  based  on 
the  results  of  the  PSA.  On  the  other  hiuid,  it 
might  be  expected  that  none  of  the  measures 
would  igibility  for  Consideration  (EFC),  an 
assessment  made  by  an  indqiendent  group  of 
raters. 

METHOD 

Subjeas 

Records  of  field  training  p^ormance  were 
obtained  for  1,352  applicants  to  the  SIDP  pro¬ 
gram  who  entered  the  FAA  Academy  b^een 
August  1981  and  September  1985,  successfully 
completed  that  program,  and  entered  ATCS  field 
training.  Three  records  were  discarded  because 
the  field  training  data  were  unavailable.  For 
other  records,  fidd  training  data  were 
occasionally  incompl^e,  resulting  in  some 
missing  data. 

Training  Performance  Measures 

Because  of  the  differences  in  the  amount  and 
content  of  training  provided  to  ATCSs  assigned 
to  different  types  of  facilities,  some  measures  of 
field  training  performance  should  not  be 
compared  for  the  3  types  of  air  traffic 
controllers.  The  only  available  measures  of 
training  performance  appropriate  for  assessing 
general  technical  competence  are  instructor 
ratings  and  whdher  or  not  the  controller  was 
successful  in  training  at  his  or  her  first  facility. 
(Controllers  who  are  unsuccessful  in  their  initial 
assignment  may  be  reassigned  to  a  facility  with 
lower  air  traffic  count  and  complexity.  Some 
controllers  who  failed  at  their  first  facility  could 
eventually  reach  FPL  status  at  a  subsequent 
facility  and  eventually  apply  for  the  SIDP 
program.)  However,  it  is  reasonable  to  analyze 
the  relationships  between  more  specific  measures 
of  field  training  performance,  such  as  the  amount 
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of  time  required  to  reach  FPL  status  and  times  to 
conq)lete  specific  phases  of  training  for  students 
who  were  assigned  to  comparable  facilities.  Such 
analyses  were  conducted  for  controllers  assigned 
to  en  route,  terminal  approach  control,  and  tower 
cab  facilities. 

RESULTS 

Results  will  be  discussed  in  the  context  of 
research  questions  that  address  factors  which  are 
relevant  for  different  groupings  of  controllers. 

Question  1:  For  the  combined  group  of  all 
controllers,  how  does  proficiency  in  technical 
training  relate  to  success  in  SIDP? 

The  only  measures  of  technical  training 
performance  relevant  for  all  controllers  are 
instructor  ratings  and  overall  training  success. 
Table  1  shows  the  relationship  between  instructor 
ratings  and  status  in  the  different  stages  of  die 
SIDP  process.  Those  referred  to  the  SBI  had 
significantly  higher  instructor  ratings  than  those 
not  referred,  /(1217)=  2.37,  p  <  .02,  and  those 
selected  to  be  supervisors  had  significandy  higher 
instructor  ratings  than  those  not  selected,  r(672) 
=  2.46,  p  <  .02.  However,  there  was  no 
difference  in  the  instructor  ratings  as  a  function 
of  SBI  eligibility  for  consideration  status. 

Table  2  shows  the  relationship  between 
training  success  and  SIDP  status.  The  likelihood 
of  being  successful  in  training  was  indqiendent 
of  referral  to  the  SBI,  x*(l)  =  .0005,  p  >  .98, 
and  was  also  indqiendent  of  eligibility  for  con¬ 
sideration,  x:*(l)  =  1.98,  p  >  .15.  However, 
training  success  was  related  to  supervisory 
selection,  x*(i)  =  5.54,  p  <  .02.  A  higher 
percentage  of  those  who  were  selected  for 
supervisory  positions  were  successful  in  training 
at  their  first  facility  than  those  who  were  not 
selected. 

Logistic  regression  analyses  were  conducted 
to  assess  the  relative  contribution  of  both 
instructor  rating  and  training  success  in 
predicting  status  in  the  3  stages  of  the  SIDP 
program.  Using  a  forward  selection  procedure, 
it  was  found  that  adding  the  mean  instructor 
rating  to  a  model  containing  no  variables  was  a 


significant  improvement  in  predicting  referral  for 
the  Skill  Based  Interview,  x^(l)  =  5.07,  p  < 
.03,  while  adding  training  status  to  that  model 
provided  no  significant  improvement,  x^(l)  = 
.18,  p  >  .65.  Neither  mean  instructor  rating  nor 
training  status  contributed  significantly  to  the 
prediction  of  eligibility  for  consideration.  Adding 
training  status  to  a  model  containing  no  vari^les 
significantly  improved  the  prediction  of 
sup^isory  selection,  x\l)  =  9.02,  p  <  .03, 
but  adding  mean  instructor  rating  to  Aat  model 
did  not  add  significantly  to  the  prediction,  x^(l) 
=  2.44,  p  >  .10. 

It  must  be  mentioned  that  the  goodness  of  fit 
of  the  model  is  somewhat  questionable;  the  -2 
Log  Likelihood  (-2  LL)  statistics,  indicating  how 
well  the  model  fits  the  data,  were  significantly 
different  from  1  for  the  prediction  of  SBI 
Referral  and  supervisory  selection;  -2  LL  (SBI 
referral)  =  1344.55,  p  <  .01,  df^  1218,  '«nd 
-2  LL(supervisory  selection)  =  752.49,  p  <  .02, 
=  673.  However,  the  goodness  of  fit  statistics 
for  the  same  models  did  not  lead  to  the  rejection 
of  the  models;  Goodness  of  fit  (SBI  referral)  = 
1219.0,  p  >  .48,  df  =  1218,  and  Goodness  of 
fit  (supervisory  selection)  =  674.0,  p  >  .48,  df 
=  673.  The  prediction  of  eligibility  for  con¬ 
sideration  by  the  logistic  regression  model 
resulted  in  a  classification  of  all  iqiplicants  as 
"eligible,”  and  the  prediction  of  selection  by  the 
model  resulted  in  a  classification  of  all  candidates 
as  "not  selected."  Thus,  while  the  model 
successfully  predicted  EFC  and  supervisory 
sdection  in  a  statistical  sense,  the  resulting 
classifications  predicted  by  the  model  are  of 
questionable  utility. 

Question  2:  For  en  route  controllers,  do  any 
of  the  specific  measures  of  training  perfor¬ 
mance  pi^ict  success  in  SIDP? 

The  measures  of  technical  training  p^or- 
mance  relevant  to  the  analyses  concerning  the  en 
route  option  included  time  to  reach  FPL  status 
(in  years),  the  number  of  calendar  days  and  OJT 
hours  required  to  complete  training,  and  mean 
instructor  ratings  in  OJT.  Table  3  shows  means 
and  standard  deviations  for  the  field  training 
performance  measures  by  status  in  each  stage  of 
the  SIDP  program.  Those  referred  to  the  SBI  had 


significantly  higher  instructor  ratings  than  those 
not  referred.  (F(l,456)  =  14.3,  p  <.001),  and 
used  significantly  fewer  QJT  hours  to  complete 
training  than  did  those  not  referred  (fX 1,453)  = 
4.49,  p  <  .04).  No  differences  in  field  training 
porfonnance  as  a  function  of  eligibility  for 
considoation  were  obsnved.  Those  sdected  as 
supervisors  spent  significantly  fewer  years  in 
training  1^X1,239)  =  7.65,  p  <  .01),  and 
significantly  fewer  days  in  OJT,  (fXi>243)  = 
6.34,  p  <  .02),  than  did  those  not  selected. 

Logistic  r^ession  analyses  were  conducted 
to  assess  die  relative  contribution  of  the  training 
measures  in  predicting  status  in  each  phase  of 
SIDP.  A  model  was  developed  to  predict  SIDP 
status,  using  die  following  as  indqiendent  vari¬ 
ables:  the  number  of  hours  and  calendar  days 
used  to  conqilete  OJT  on  the  first  2  sectors, 
mean  instructor  ratinp  for  OJT  phases,  and  the 
amount  of  time  (in  years)  to  reach  FPL  status.  A 
model  containing  mean  instructor  rating  sig- 
nificandy  predicted  referral  to  the  SBI  (x^(l)  = 
16.5,  p  <  .001)  while  the  other  variables  con¬ 
tribute  nothing  more  to  die  prediction  of  the 
model  (x*(3)  =  1.95,  p  >  .58).  None  of  the 
variables  contributed  significandy  to  a  model 
predicting  eligibility  for  considoation  (x2(4)  » 
6.61,  p  >  .15).  A  modd  containing  only  the 
time  to  reach  I7L  status  significandy  predicted 
supervisory  selection  (x^(l)  =  7.20,  p  <  .01), 
while  die  other  variables  contributed  nodiing  to 
the  prediaion  of  the  model  (x*(3)  =  4.16,  p  > 
.24). 

It  must  be  noted  that  the  goodness  of  fit  of 
the  model  is  somewhat  questionable.  The  -2  LL 
statistic,  indicating  how  well  the  model  fits  the 
data,  was  significandy  different  from  1  for  the 
prediction  of  SBI  referral  status  (-2  LL  = 
488. 15,  p  <  .04,  df  =  435)  but  the  goodness 
of  fit  statistics  for  the  same  model  did  not  lead 
to  the  rejeaion  of  the  model  (Goodness  of  fit  = 
436.0,  p  >  .47,  df  =  435.)  The  prediction  of 
eligibility  for  consid«ation  by  the  logistic 
regression  model  resulted  in  a  classification  of 
all  but  2  iqiplicants  as  "eligible."  Although  both 
goodness  of  fit  statistics  for  the  model  predicting 
supervisory  selection  suggested  that  the  model  fit 
the  data,  the  prediction  of  selection  by  the  model 
resulted  in  a  classification  of  all  c^idates  as 


"not  selected."  Thus,  while  the  model  successful¬ 
ly  predicted  EFC  and  supervisory  selection  in  a 
statistical  sense,  the  classifications  predicted  by 
the  model  are  of  questionable  utility. 

Question  3:  For  en  route  controllers,  in  what 
ways  are  the  measures  of  training  per¬ 
formance  related  to  the  rating  dimoisions  used 
to  determine  success  in  SIDP? 

Table  4  shows  descriptive  statistics  for  the 
predictors  and  rating  dim^ions  for  en  route 
controllers,  and  Table  5  shows  correlations 
between  the  predictors  and  the  SIDP  rating 
dimensions.  Several  of  the  measures  of  training 
performance  had  statistically  significant,  though 
somewhat  low,  correlations  widi  the  PSA  ratings, 
and  had  virtually  no  relationship  with  ratings 
from  the  SBI. 

Regression  analyses,  with  results  shown  in 
Table  6,  were  conducted  to  assess  the  relative 
contribution  of  the  training  measures  in  predict¬ 
ing  each  of  the  rating  dimensions  in  the  SIDP 
program.  While  the  training  performance  mea¬ 
sures  were  significantly  related  to  ratings  made 
on  PSA  dimmisions,  v^  little  relationship  was 
observed  b^een  training  measures  and  ratings 
made  on  the  SBI.  OJT  hours  ami  the  number  of 
days  in  OJT  were  significantly  related  to  the 
composite  score,  and  the  ^mmunications. 
Interpersonal  and  Leadership  rating  dimensions 
of  the  PSA.  A  model  containing  both  OJT  time 
and  instructor  rating  predicted  the  PSA  Technical 
rating  dimension.  However,  most  of  the  SBI 
rating  dimensions  were  not  predicted  by 
measures  of  training  performance.  The  number 
of  years  to  reach  FPL  status  had  a  modest 
relationship  with  ratings  on  Knowledge  of 
Supervisory  Role  Performance;  OJT  time  had  a 
nu^est  relationship  with  the  ratings  on 
Decisiveness.  Training  performance  measures 
accounted  for  10%  or  less  of  the  variance  in  the 
PSA  and  SBI  ratings. 

(Question  4:  For  terminal  radar  approach 
controllers,  do  any  of  the  specific  measures  of 
training  performance  predict  success  in  SIDP? 

The  measures  of  technical  training  perfor¬ 
mance  relevant  to  the  analyses  dealing  with 
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tttinina]  radar  approach  controllers  included  time 
to  reach  FPL  status  (in  years),  calendar  days  and 
OJT  hours  required  to  conq>lete  radar  training, 
and  an  instructor  rating  in  radar  training.  The 
training  days  and  OJT  hours  were  adjusted  by 
dividing  diem  by  die  number  of  radar  positions 
in  the  airspace  for  which  their  facility  was  re¬ 
sponsible. 

Table  7  shows  means  and  standard  deviations 
for  the  field  training  performance  measures  by 
status  in  each  stage  of  die  SIDP  program.  Those 
referred  to  the  SBI  took  significandy  fewer  years 
to  reach  FPL  status  than  did  those  who  were  not 
referred,  ^1(1,314)  =  4.33,  p  <  .04.  Logistic 
regression  analyses  were  tfaoi  conducted  to  assess 
the  rdative  contribution  of  the  training  measures 
in  predicting  status  in  each  phase  of  SIDP. 
A  model  was  developed  to  predict  SIDP  status, 
using  the  measures  described  above  as  indqiien- 
dent  variables  and  the  3  stages  of  advancement 
in  SIDP  as  the  dependent  variables.  A  model 
containing  time  to  readi  FPL  status  marginally 
predicted  referral  to  the  Skill-Based  Interview, 
X*(l)  =  3.74,  p  <  .06,  while  instructor  rating, 
OJT  hours,  and  number  of  days  spent  in  radar 
training  did  not  contribute  significantly  to  the 
model,  x*(3)  =  4.09,  p  >  .25.  The  training 
variables  did  not  contribute  significantly  to  the 
prediction  of  digibility  for  consideration,  nor  did 
diey  contribute  to  the  prediction  of  supervisory 
selection. 

Although  both  goodness  of  fit  statistics  for 
the  model  predicting  referral  for  the  SBI 
suggested  that  the  model  fit  the  data  (-2  LL  = 
303.57,  p  >  .26,  df  =  289),  the  prediction  of 
eligibility  by  the  model  resulted  in  a  classification 
of  all  ci^idates  as  "eligible."  Thus,  while  the 
model  successfolly  predicted  EFC  status  in  a 
statistical  sense,  the  classifications  are  not  useful. 

Question  5:  For  terminal  radar  approach 
controllers,  in  what  ways  are  the  measures  of 
training  performance  related  to  the  rating 
dimensions  used  to  determine  success  in  SIDP? 

Table  8  shows  descriptive  statistics  for  ail 
predictors  and  criteria.  Table  9  shows  correla¬ 
tions  between  the  predictors  and  SIDP  rating 
dimensions.  The  number  of  years  to  attain  FPL 


status  was  significantly  correlated  with  several 
of  the  PSA  rating  dimensions,  but  only  with  the 
Supervisory  Role  rating  dimension  of  foe  SBI  (in 
foe  opposite  direction  than  was  predicted).  OJT 
hours  also  had  significant  negative  correlations 
with  foe  PSA  rating  dimensions  but  were  posi¬ 
tively  correlated  with  foe  Judgment  rating  dimen¬ 
sion  of  foe  SBI. 

Regression  analyses,  shown  in  Table  10,  were 
conducted  to  assess  foe  relative  contribution  of 
foe  training  measures  in  predicting  eadi  of  foe 
rating  dimensions  in  foe  SIDP  program.  Mea¬ 
sures  of  technical  training  performance  were 
again  more  highly  related  to  ratings  made  by 
peers  and  supervisors  than  ratings  made  by  those 
conducting  foe  Skill-Based  Int^iew.  For  radar 
ai^roach  controlla*  candidates,  OJT  hours  sig¬ 
nificantly  predicted  ratings  on  all  foe  PSA  scales 
excq[)t  for  foe  Technical  scale,  which  was  predic¬ 
ted  by  a  model  containing  both  instructor  ratings 
and  time  to  complete  training.  Regression  modds 
containing  OJT  hours  also  predicted  ratmgs  on 
foe  SBI  Judgment  and  Decisiveness  scales,  but 
foe  relationship  was  not  in  foe  predicted  direc¬ 
tion.  None  of  the  models  containing  measures  of 
training  performance  accounted  for  more  foan 
10%  of  foe  variance  in  foe  ratings. 

Question  6:  For  tower  cab  controllers,  do  any 
of  the  specific  measures  of  training 
performance  predict  success  in  SIDP? 

The  measures  of  technical  training  pofor- 
mance  relevant  to  foe  analyses  dealing  with  town’ 
cab  controllers  included  time  to  reach  FPL  status 
(in  years),  calendar  days  and  OJT  hours  required 
to  complete  training  on  foe  local  control  position, 
and  foe  mean  instructor  rating  for  OJT.  Those 
referred  to  foe  SBI  h^  significantly  higher 
instructor  ratings  foan  did  foose  not  referred, 
J'X1,230)  =  5.14,  p  <  .03.  No  significant 
differoices  in  field  training  performance  were 
observed  as  a  function  of  eligibility  for 
consideration  or  supervisory  sdection.  Logistic 
regression  analyses  were  conducted  to  assess  foe 
relative  contribution  of  foe  training  measures  in 
predicting  status  in  each  phase  of  SIDP.  A  model 
was  developed  to  predict  SIDP  status,  using  foe 
measures  described  above  as  independent 
variables  and  status  in  foe  3  stages  of  SIDP  as 
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the  dependent  variables.  None  of  the  measures  of 
field  training  performance  sufficiently  predicted 
any  stage  of  the  SIDP  process  to  justify  includ¬ 
ing  th^  in  a  model. 

Question  7:  For  tower  cab  controllers,  in  what 
ways  are  the  measures  of  training  per¬ 
formance  related  to  the  rating  dimensions  used 
to  determine  success  in  SIDP? 

Table  12  shows  descriptive  statistics  for  all 
predictors  and  criteria.  Table  13  shows  correla¬ 
tions  of  the  predictors  with  PSA  and  SBI  rating 
dimrasions.  The  only  significant  corrdations 
were  between  the  training  measures  and  the 
C^ganizing/Planning  rating  dimension  of  the  SBI. 
Regression  analyses,  shown  in  Table  14,  were 
conducted  to  assess  the  relative  contribution  of 
the  training  measures  in  predicting  each  of  the 
naing  dimensions  in  the  SIDP  program.  For 
tower  cab  controllers,  none  of  the  training  pec- 
formance  measures  significantly  predicted  the 
PSA  scales,  but  the  number  of  years  required  to 
reach  FPL  status  predicted  ratings  on  both  the 
Organizing/Ptanning  and  the  Direction/Motiva¬ 
tion  SBI  rating  dimensions.  However,  the  per¬ 
centage  of  variance  in  the  ratings  accounted  for 
by  the  number  of  years  to  reach  FPL  status  was 
less  than  S%. 

DISCUSSION 

The  purpose  of  diis  study  was  to  assess  the 
contribution  of  measures  of  ATCS  field  training 
p^ormance  in  predicting  1)  outcomes  in  the 
multiple  hurdles  of  the  Sup^isory  Identification 
and  Development  Program,  2)  ratings  made  by 
peers  and  supervisors,  and  3)  ratings  of  know¬ 
ledges  and  skills  made  by  a  panel  during  role 
plays. 

Descriptive  statistics  were  computed  and 
regression  and  logistic  regression  analyses  were 
conducted  for  all  terminal  and  en  route  con¬ 
trollers  combined.  The  Instructor  rating  in  on- 
the-job  training  was  related  to  referral  for  the 
SBI  and  training  status  was  rdated  to  supervisory 
selection. 

Analyses  were  also  conducted  separately  for 
en  route,  terminal  radar  ^proadi  control,  and 


tower  cab  air  traffic  controllers  because  they 
perform  different  job  functions  and  undergo 
different  types  of  training.  The  measures  of  field 
training  peiformance  were  found  to  contribute  in 
different  ways  to  the  prediction  of  success  in 
SIDP  for  the  controllers  in  different  options.  In 
the  en  route  option,  instructor  ratings  were 
related  to  referral  to  die  SBI,  and  the  amount  of 
time  required  to  reach  FPL  status  was  related  to 
supervisory  selection.  None  of  the  training 
measures  predicted  eligibility  for  consideration  as 
a  supervisor.  For  terminal  radar  iqiproadi 
controllers,  the  number  of  years  required  to 
reach  FPL  status  was  related  to  referral  to  the 
SBI,  but  none  of  the  training  measures  predicted 
eligibility  for  consideration  or  actual  selection. 
For  tower  cab  controllers,  none  of  the  training 
measures  contributed  significandy  to  the 
prediction  of  status  in  SIDP.  In  all  cases,  the 
logistic  regression  analyses  produced  modds 
whidi  predicted  that  all  applicants  would  be 
referred  for  consideration  and  no  applicants 
would  be  selected.  Thus,  while  statistically 
significant  results  were  observed,  no  meaningful 
distinction  among  candidates  was  found. 

Additional  regression  analyses  w^e  conduc¬ 
ted  which  included  the  technical  training  perfor¬ 
mance  measures  that  were  specific  to  each  con¬ 
troller  option.  For  en  route  and  terminal  radar 
approach  controllers,  the  results  suggested  that 
ca^idates’  technical  proficimcy  demonstrated  in 
training  was  related  to  subsequent  peer  and 
supervisor  ratings  of  technical  competence.  The 
measures  of  technical  proficiency  contributed  to 
the  prediction  of  ratings  made  by  peers  and 
supervisors  who  were  aware  of  the  candidates’ 
technical  skills.  On  the  other  hand,  those  who 
rated  candidates  during  the  SBI  did  not  rate  nor 
were  th  n*  familiar  with  the  candidates’  technical 
proficiency.  Correspondingly,  the  measures  of 
candidates’  technical  training  performance  were 
generally  not  significantly  or  highly  predictive  of 
foeir  SBI  ratings.  This  outcome  is  as  expected; 
the  purpose  of  the  SBI  is  to  reinforce  selection 
on  foctors  other  than  technical  competence  alone. 

Furthermore,  in  examining  more  closely  the 
correlations  between  measures  of  training  p^or- 
mance  and  PSA  ratings,  the  measures  of  die 
amount  of  time  required  to  complete  certain 
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training  phases  predicted  ratings  on  scales  pres¬ 
umably  unrelated  to  tedinical  skills—leadership, 
communication,  and  interpersonal  skills.  This 
finding  suggests  that  p^cq>tions  of  technical 
proficiency  may  bias  how  peers  and  supervisors 
interpret  other  aspects  of  onployees’  perfor¬ 
mance,  and  subsequently  influence  their  inter¬ 
pretation  of  supervisory  potential.  On  the  other 
hand,  the  rating  dimensions  may  tap  overlapping 
characteristics;  for  example,  peers  and 
supervisors  may  interpret  le^ership  to  include 
technical  components,  thus  resulting  in  significant 
correlations  between  training  performance  and 
ratings  made  on  other  dimensions. 

Moreover,  measures  of  technical  proficiency 
and  instructor  ratings  both  contributed  to  the 
prediction  of  ratings  made  on  the  PSA  Technical 
scale,  although  the  instructor  rating  did  not 
contribute  significantly  to  regression  models 
predicting  ratings  made  on  any  of  the  other  PSA 
scales.  This  result  might  suggest  that  some  factor 
other  than  performance  alone  contributes  to  the 
assessment  of  technical  proficiency,  but  such  a 
speculation  is  not  confirmed  by  observing 
particularly  high  corrdations  b^een  instructor 
ratings  and  other  PSA  rating  scales  that  might 
measure  fob  other  factor  (sudi  as  communication 
or  interpmonal  skills). 

Finally,  similar  results  were  not  observed  for 
tower  cab  controllers.  One  reason  might  be  that 
insufficient  numbers  of  candidates  had  trained  at 
tower  cab  facilities  as  their  first  facility.  Also,  it 
is  possible  that  foe  measures  of  training 
performance  do  not  describe  technical  proficiency 
for  foe  tower  cab  controller  as  well  as  they  do 
for  other  types  of  controllers  vfoo  spend  longer 
periods  of  time  in  training. 

In  summary,  available  measures  of  field 
training  performance,  though  hindered  by  mea¬ 
surement  problems,  are  somewhat  predictive  of 
both  ratings  of  future  potential  for  success  as  a 
first-line  supervisor  and  of  successfully 
complying  foe  SIDP  program,  a  prerequisite  for 
becoming  a  first-line  supervisor.  In  one  sense, 
foe  result  is  somewhat  predictable,  because  at 
least  minimal  technic^  competence  is  a 
requirement  for  a  first-line  supervisor  (however, 
see  Slusher,  Van  Dyke,  &  Rose,  1972). 


Although  Della  Rocco  &  Broach,  (1991) 
suggested  that  technical  proficiency  measured  in 
the  FAA  Academy  was  not  predictive  of  SIDP 
outcomes,  the  training  performance  measures 
used  here  occurred  in  a  setting  that  resembled  foe 
job  more  closely  than  did  the  ATC  screen 
program,  and  foe  measures  were  obtained  more 
recently  than  were  foe  screen  performance 
measures. 

On  the  other  hand,  foe  results  are  somewhat 
unexpected,  because  a)  foe  training  performance 
measures  are  fairly  global  ami  do  not  describe 
specific  strengths  or  weaknesses  in  performance, 
b)  instructor  ratings  were  usually  made  by 
personnel  untrained  in  foe  use  of  foe  rating  scales 
(and  thus,  were  very  likely  tainted  by  a  number 
of  rater  biases),  c)  foe  measures  were  obtained 
during  training  and  reflect  skill  learning  rather 
than  job  performance,  and  d)  a  number  of  ofoer 
factors  besides  technical  success  are  assessed  in 
determining  success  in  foe  SIDP  program.  At  foe 
same  time,  it  must  be  remembered  that  none  of 
the  measures  of  training  performance  accounted 
for  more  than  10%  of  foe  variance  in  ratings  of 
supervisory  potential  made  for  candidates  to  foe 
SIDP  program.  At  best,  these  measures  might  be 
added  to  regression  models  containing  other 
variables  to  provide  additional  contribution  to  foe 
prediction  of  SIDP  ratings. 

It  is  not  suggested  that  foe  available  measures 
of  field  training  performance  sufficiently  describe 
controller  performance.  A  future  research  study 
is  planned  to  identify  and  devdop  better  mea¬ 
sures  of  performance  during  training  and  on  foe 
job.  When  foose  measures  are  available,  it  will 
be  interesting  to  return  to  foe  question  of  foe 
relationship  between  technical  performance  and 
supervisory  success  and  determine  whetho'  foe 
relationships  found  in  this  study  are  still  rele¬ 
vant. 
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TABLE  1 

Relationship  between  overall  Instructor  rating  and  SIDP  status  for  all  ATC  options 


SIDP  Status 

M 

SD 

N 

Referred 

4.09 

.87 

926 

Not  referred 

3.96 

.85 

293 

Eligible 

4.09 

.86 

674 

Not  eligible 

4.11 

.88 

248 

Selected 

4.23 

.81 

166 

Not  selected 

4.04 

.88 

508 

TABLE  2 

Draining  success  at  first  facility  and  SIDP  status  for  all  ATC  options 


% 

WWt 

SIDP  Status 

Successful 

Successful 

N 

Referred 

82.1 

17.9 

1018 

Not  referred 

82.2 

17.8 

331 

Eligible 

83.1 

16.9 

739 

Not  eligible 

79.3 

20.7 

275 

Selected 

88.8 

11.2 

179 

Not  selected 

81.3 

18.8 

560 
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Tables 

Relationship  between  field  training  performance  and  status  in  SIDP  En  Rome  controllers 
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Table  4 

Descriptive  statistics  for  predictors  and 
SIDP  rating  dimensions  for  En  Route  controllers 


Measures 

M 

SD 

N 

Training  Performance 

Instructor  Rating 

4.25 

.53 

458 

Yews  to  reach  FPL 

2.60 

.59 

442 

OJT  Hours 

235.33 

88.68 

455 

OJT  Days 

142.17 

81.67 

454 

PSA  Ratings 

Composite 

61.04 

15.52 

467 

Communication 

62.01 

15.34 

466 

Interpersonal 

60.34 

17.57 

466 

Leadership 

60.22 

16.91 

466 

Technical 

61.59 

15.76 

466 

SBI  Ratings 

Agency  Programs 

3.66 

.83 

313 

Supervisory  Role 

3.84 

.81 

313 

Problem  Solv/ Analytical 

3.96 

.89 

313 

Judgment 

3.82 

.94 

313 

Decisiveness 

4.07 

.87 

313 

Organizing/Planning 

3.93 

.91 

313 

Interpersonal  Skill 

4.21 

.85 

313 

Communication 

4.06 

.89 

313 

Direction/Motivation 

3.94 

.93 

313 
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Tables 

Correlations  between  SIDP  rating  dimensions 
and  training  measures  for  En  Route  controllers 


Training  Measures 

SIDP  Ratings 

Years  to 

FPL  Status 

Instructor  OJT 

Rating  Hours 

OJT 

Days 

PSA  Ratings  (N=435) 

Con^site 

-.10’ 

.24“ 

-.20“ 

-.18“ 

Conununication 

-.08 

.21“ 

-.18“ 

-.15“ 

Impersonal 

-.06 

.19“ 

-.16“ 

-.16“ 

Leadership 

-.11* 

.22“ 

-.20“ 

-.17“ 

Technical 

-.14“ 

.31“ 

-.20“ 

-.22“ 

SBI  Ratings  (N=291) 

Agency  Programs 

.01 

.02 

-.08 

.00 

Supervisory  Role 

-.13* 

.01 

-.01 

-.02 

Prob  Solv/Analytical 

.07 

-.05 

.00 

.00 

Judgment 

.08 

-.05 

-.07 

-.09 

Decisiveness 

.02 

.04 

-.06 

-.11 

Organizing/Planning 

.09 

-.06 

-.04 

-.05 

Impersonal  Skill 

-.01 

-.05 

.02 

-.08 

Communication 

.03 

-.01 

-.03 

.08 

Direction/Motivation 

.05 

-.09 

.00 

.06 

*p  :<  .01  **p  <.  .001 


36 


Table  6 

Results  of  regression  analyses  using 
field  training  performance  measures  as  predictors  of 
component  rating  dimension  scores  in  SIDP  En  Route  controllers 


Critwion 

Predictors 

R 

/3 

F 

P 

PSA  Ratings 

Composite 

OJT  Hours 

.23 

.05 

-.23 

16.13 

.0001 

Communication 

OJT  Hours 

.18 

.03 

-.18 

9.35 

.003 

Intnpersonal 

OJT  Hours 

.18 

.03 

-.18 

9.57 

.003 

Leadmhip 

QIT  Hours 

.24 

.06 

-.24 

16.91 

.0001 

Technical 

OJT  Hours 

.32 

.10 

-.25 

16.60 

.0001 

Inst  Rating 

-.15 

SBI  Ratings 

Agoicy  Programs 

ns 

Supervisory  Role 

Years  to  FPL 

.14 

.02 

-.14 

5.35 

.03 

Problem  Solving 

ns 

Judgment 

ns 

Decisivoiess 

OJT  Hours 

.12 

.01 

-.12 

4.14 

.05 

Organizing 

ns 

Interptf  Skill 

ns 

Communication 

ns 

Direction/Motiv 

ns 
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Table  7 

Relationship  between  field  training  performance  and  status  in  SIDP  Terminal  Radar  Approach  controllers 
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Tables 

Descriptive  statistics  for  predictors  and  SIDP 
rating  dimensions  for  Terminal  Radar  Approach  controllers 


Measures 

M 

SD 

N 

Instructor  Rating 

Training  Performance 

4.04  .84 

368 

Years  to  readi  FPL 

1.84 

.87 

316 

OUT  Hours 

62.04 

40.60 

338 

OTT  Days 

95.06 

66.65 

339 

Cra^osite 

51.66 

PSA  Ratings 

13.77 

378 

Communication 

59.49 

13.46 

377 

Interpersonal 

57.58 

15.32 

378 

Leadership 

56.38 

15.74 

378 

Tedmical 

57.38 

15.42 

378 

Agency  Programs 

3.84 

SBI  Ratings 

.79 

270 

Supervisory  Role 

3.88 

.76 

270 

Problem  Solv/Analytical 

4.00 

.89 

270 

Judgm^ 

3.84 

.89 

270 

Decisiveness 

4.11 

.91 

270 

Organizing/Planning 

3.98 

.89 

210 

InterpOTonal  Skill 

4.24 

.83 

270 

Communication 

4.13 

.80 

270 

Direction/Motivation 

3.96 

.91 

270 
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Table  9 

Correlations  between  SIDP  rating  dimensions 
and  training  measures  for  Terminal  Radar  Approach  controllers 


Training  Measures 


SIDP  Ratings 

Years  to 

FPL  Status 

Instructor 

Rating 

OJT 

Hours 

OJT 

Days 

Composite 

PSA  Ratings  (N  =  304) 

-.12*  .03 

-.15" 

-.10 

Communication 

-.07 

.00 

-.15" 

-.06 

Intopersonal 

-.03 

-.03 

-.14* 

-.06 

Leadership 

-.13* 

.02 

-.14* 

-.10 

Technical 

-.21" 

.09 

-.12* 

-.14* 

Agency  Programs 

SBI  Ratings  (N  =  220) 

.05  -.00 

.02 

.00 

Supervisory  Role 

.13 

.09 

-.06 

-.05 

Prob  Solv/An^ytical 

.02 

-.04 

.11 

.07 

Judgment 

.05 

-.04 

.15* 

.05 

Decisiveness 

.03 

-.03 

.08 

-.01 

Organizing/Planning 

.06 

-.06 

.13 

.09 

Interpersonal  Skill 

.01 

-.07 

.12 

.05 

Communication 

.06 

-.10 

.08 

.02 

Direction/Motivation 

.01 

-.07 

.05 

-.01 

*P<.  .05  **p  ^  .01 
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Table  10 

Results  of  regression  analyses  using  field  trainings 
performance  measures  as  prediaors  of  component  rating 
dimension  scores  in  SIDP  Terminal  Radar  Approach  controllers 


Crit»k)ii 

Predictors 

6 

F 

P 

PSA 

Ratings 

C(»iq)osite 

OIT  Hours 

.24 

.06 

-.24 

12.66 

.001 

Coii^nication 

OJT  Hours 

.21 

.05 

-.21 

9.95 

.002 

Interpenonal 

QIT  Hours 

.23 

.05 

-.23 

11.38 

.001 

Leadership 

OIT  Hours 

.19 

.04 

-.19 

8.14 

.005 

Tedmical 

Inst  Rating 

.28 

.08 

.18 

8.68 

.001 

OJT  Days 

-.17 

SBI 

Ratings 

Agency  Programs 

ns 

Supervisory  Role 

ns 

Problem  Solving 

ru 

Judgmait 

OIT  Hours 

.16 

.02 

.16 

5.30 

.03 

Decisiveness 

ns 

Organizing 

OJT  Hours 

.14 

.02 

.14 

3.97 

.05 

Intoper  Skill 

ns 

Communication 

ns 

Direction/Motiv 

ns 
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Table  11 

Relationship  between  field  training  performance  and  status  in  SIDP  Tower  Cab  controllers 
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Table  12 

Descriptive  statistics  for  predictors  and 
SIDP  rating  dimensions  for  Tower  Cab  controllers 


Measures 

M 

SD 

N 

Instructor  Rating 

Training  Performance 

4.23  .90 

232 

Years  to  readi  FPL 

.66 

.35 

240 

OTT  Hours 

104.17 

44.81 

238 

OJT  Days 

162.65 

103.38 

247 

Con^osite 

57.12 

PSA  Ratings 

15.52 

251 

Communication 

59.10 

15.88 

251 

Interpersonal 

56.84 

17.92 

251 

Leadership 

55.65 

17.18 

251 

Technical 

56.83 

16.62 

251 

Agency  Programs 

3.49 

SBI  Ratings 

.88 

162 

Supervisory  Role 

3.75 

.86 

162 

Problem  Solv/ Analytical 

3.85 

1.02 

162 

Judgment 

3.66 

.93 

162 

Decisiv^iess 

3.83 

1.04 

162 

Organizing/Planning 

3.75 

1.01 

162 

Interpersonal  Skill 

4.05 

.87 

162 

Communication 

3.88 

.99 

162 

Direction/Motivation 

3.76 

.98 

162 
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Table  13 

Correlations  between  SIDP  rating  dimensions 
and  training  measures  for  Tower  Cab  controllers 


Training  Measures 


Years  to  Instructor  OJT  OJT 

SIDP  Ratings  FPL  Status  Rating  Hours  Days 


PSA  Ratings 


Composite 

.01 

.08 

-.00 

.01 

Communication 

-.01 

.06 

.01 

-.01 

Interpersonal 

-.01 

.07 

.03 

.02 

Leadership 

.01 

.08 

-.00 

-.00 

Technical 

.05 

.11 

-.05 

.03 

SBI  Ratings 

Agency  Programs 

.01 

.01 

-.07 

-.04 

Supervisory  Role 

-.09 

.13 

.03 

-.05 

Prob  Solv/ Analytical 

-.15 

.11 

-.09 

-.11 

Judgment 

-.08 

.07 

-.04 

-.10 

Decisiveness 

-.14 

.03 

-.09 

-.09 

Organizing/Planning 

-.20* 

.13 

.18* 

-.17* 

Interpersonal  Skill 

-.15 

.12 

-.08 

-.06 

Communication 

-.15 

.11 

-.04 

-.06 

Direction/Motivation 

-.ir 

.06 

-.12 

-.13 

*P  .<  -01 

V| 

II 
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Table  14 

Results  of  regression  analyses  using 
field  training  peiformance  measures  as  prediaors  of 
component  rating  dimension  scores  in  SIDP  Tower  Cab  controllers 


Critnion 


Predictors 


Agoicy  Programs 
Supervisory  Role 
Problem  Solving 
Jud^n^ 
Demivowss 


SBI  Ratings 


Organizing 

Interper  Sul 

Years  to  FPL 
ns 

.20 

.04 

-.20 

6.12 

.02 

ComuHinication 

ns 

Direction/Motiv 

Years  to  FPL 

.17 

.03 

-.17 

4.04 

.05 

CANDIDATE  PERFORMANCE  IN  A  SUPERVISORY  SELECTION  PROGRAM 
AND  SUBSEQUENT  SELECTION  DECISIONS 


Jennifer  G.  Myers,  Ph.D. 


SmaJl  surveys  of  private  organizations  have 
reported  that  fewer  than  half  of  these  organiza¬ 
tions  have  formal  selection  systems  in  place  for 
first-line  supervisors  (Levine,  1986;  Rendero, 
1980).  Given  the  importance  of  first-line  super¬ 
visors  in  insuring  employee  productivity  by 
directing  subordinate  work  activity,  as  well  as 
their  role  in  rqiresenting  management  and  the 
organizational  mission  to  subordinates,  the  lack 
of  rqwrted  formal  supervisory  selection  systems 
is  somewhat  surprising.  Recent  research  on 
supervisory  selection  systems  and  promotion 
decisions  is  also  noticeably  absent. 

A  recent  review  of  federal  agencies’  super¬ 
visor  selection  programs  (U.  S.  Merit  Systems 
Protection  Board,  1989)  described  the  typical 
situation.  In  general,  applicants  to  first-line 
supervisory  jobs  document  training  and  exper¬ 
ience  information  typically  based  on  their  prior 
nonsupervisory  work  history.  As  noted  in  the 
review,  the  problem  for  the  selecting  official 
becomes  one  of  evaluating  an  individusd’s  ability 
or  potential  for  performing  supervisory  tasks  that 
have  never  been  done  before.  Given  the  makeup 
of  the  application  materials,  selecting  officials 
primarily  see  the  applicant’s  prior  technical  work 
experience.  Personal  interviews  with  i^plicants 
may  be  restricted  by  the  associated  costs,  since 
Merit  Promotion  Plan  regulations  require  all  or 
none  of  the  eligible  candidates  to  be  interviewed. 
The  frequent  result  in  the  federal  government  is 
that  the  most  highly  technically  qualified  person 
is  chosen  for  the  supervisory  job,  rather  than 
someone  with  demonstrated  management  skills. 
This  outcome  is  similar  to  that  found  in  private 
industry  (Northrup,  Cowin,  Vanden  Plas,  Ful¬ 
mer,  ^lick,  Bellace,  &  Rosenzwig,  1978). 
However,  selections  based  on  technical  perfor¬ 
mance  may  result  in  the  promotion  of  someone 
unsuitable  for  the  position,  since  technical  skills 
and  experience  are  not  necessarily  related  to 
supervisory  competence  (Phillips,  1985). 


Technical  competence  has  been  identified  as 
an  important  factor  in  the  performance  of 
supervisory  tasks  (Myers  &  Stutzman,  1991). 
However,  Myers  and  Stutzman  (1991)  found  that 
other  competencies  such  as  communications  skills 
and  interpersonal  skills  were  mentioned  more 
frequently  than  was  technical  competence  as 
important  in  successful  task  performance.  One 
m^od  of  ensuring  the  applicant  for  a  first-line 
supervisory  position  has  the  right  mix  of 
technical  and  supervisory  skills  is  to  formalize 
the  assessment  of  the  most  important 
competencies  through  a  structured  selection 
program.  Still,  selecting  officials  could 
emphasize  the  importance  of  technical  over 
supervisory  competence,  by  selectively  attending 
to  factors  in  the  work  history  that  demonstrate 
the  applicant’s  technical  competence.  As  already 
suggested  by  Phillips  (1985),  this  may  not  be  an 
optimal  strategy  for  selecting  the  best  candidate 
for  the  position.  As  one  example,  a  study  of 
engineers  (Slusher,  Van  Dyke,  &  Rose,  1972) 
found  that  the  most  highly  technically  qualified 
supervisors  were  least  likely  to  adopt  a 
managerial  role  and  in  turn,  were  lowest  in 
group  productivity  among  the  groups  studied. 

As  noted  in  the  introduction  of  this  technical 
report,  the  Federal  Aviation  Administration 
implemented  the  Air  Traffic  Supervisory 
Identification  and  Development  Program  (SIDP) 
to  change  the  emphasis  on  technical  performance 
in  promotabiiity  decisions  to  include  other  skills, 
such  as  decision-making  and  communication,  that 
reflect  supervisory  potential.  In  order  to  under¬ 
stand  how  the  different  competencies  may 
distinguish  between  successful  and  unsuccessful 
candidates  at  different  phases  of  the  SIDP, 
comparisons  were  made  of  candidate  performance 
on  SIDP  performance  measures,  as  well  as 
measures  of  technical  performance  and 
experience.  The  influence  of  technical 
performance  and  skills  measured  in  the  SIDP  on 
success  in  each  of  the  phases  was  also  assessed. 


47 


METHOD 

Subjects 

Subjects  were  985  nonsupervisory  air  traffic 
control  specialists  who  entered  the  En  Route  or 
Terminal  Screen  Program  in  the  Academy  be¬ 
tween  1981  and  1985,  ^plied  to  SIDP  between 
1985  and  1988,  and  complied  their  field  training 
in  an  En  Route  or  Terminal  facility.  The  average 
age  of  this  group  at  the  time  they  applied  to 
SIDP  was  32.14  years  and  the  average  full 
performance  level  (FPL)  experience  was  3.69 
years.  Fifteen  percent  were  female  and  85%  were 
male.  All  ^plicants  were  required  to  have  at 
least  a  fully  successful  rating  on  their  last 
performance  appraisal;  29.6%  were  rated  fully 
successful,  52.5%  were  rated  exertional,  ai^ 
17.9%  were  rated  outstanding. 

Measures 

Peer-Supervisory  Assessmera.  Applicants  to 
SIDP  identify  a  combination  of  peers  and  first- 
line  supervisors  to  complete  the  Peer-Super¬ 
visory  Assessment  (PSA).  A  paired  comparison 
approach  is  used  to  rate  rpi>canfs  against  other 
applicants  and  benchmarks  on  4  different 
dimensions;  Technical  Competence, 
Communication  Skills,  Interpersonal  Skills,  and 
Leadership.  Scores  are  coiruted  on  a  0  to  1(X) 
scale;  the  composite  PSA  score  is  a  simple 
arithmetic  mean  of  die  4  dimension  scores. 
Successful  completion  of  this  hurdle  results  in 
referral  to  the  next  step  in  the  SIDP  process,  the 
Skill-Based  Interview  (SBI). 

SkUl-Basai  Inteniew.  The  SBI  is  a  combin¬ 
ation  of  a  face-to-face  interview  and  3  role- 
plays.  Three  interviews  provide  consensus 
ratings  on  a  1  (weak)  to  5  (outstanding)  scale  on 
the  following  9  dimensions;  Organizational 
Knowledge,  Knowledge  of  Supervisory  Role 
Performance,  Organizational  Planning,  Problem- 
Solving  and  Analytical  Ability,  Judgment, 
Decision-Making,  Communication  Skills,  Inter¬ 
personal  Skills,  uid  Leadership.  Based  upon  the 
combination  of  ratings  received  during  the  SBI, 
individuals  are  determined  to  be  (1)  ”promotabie” 
and  placed  on  the  Eligible  for  Consideration 
(EFC)  list,  allowing  them  to  apply  for  first-line 
supervisor  positions,  (2)  in  n^  of  further 


skill  remediation  prior  to  placement  on  the  EFC, 
or  (3)  recommended  for  further  self-development 
and  re£q>plication  to  SIDP  at  a  later  date. 

Journeyman  Level  Technical  Performance. 
Technical  performance  at  the  journeyman  level 
was  identified  through  2  broad  measures;  (a)  the 
number  of  months  of  journeyman  levd  ex¬ 
perience  the  applicant  had  at  the  time  of  2q)plica- 
tion  to  SIDP  and  (b)  the  most  recent  performance 
appraisal  rating  prior  to  iqiplication  to  SIDP. 
Although  performance  ratings  can  actually  range 
from  1  (unacc^table)  to  5  (outstanding),  the 
program  requirement  for  a  fully  successful  (3) 
rating  to  apply  to  SIDP  restricted  the  range  to  die 
3  highest  possible  ratings. 

Two  different  sets  of  analyses  were  con¬ 
ducted  to  examine  3  different  groups;  (a)  whether 
the  individual  was  referred  to  the  SBI  or  not 
{Referral),  (b)  whether  or  not  the  applicant  was 
placed  on  the  Eligible  for  Consideration  list 
{Promotability),  and  (c)  whether  or  not  the 
person  was  selected  for  a  first-line  supervisor 
position  (Selection).  Multivariate  analysis  of 
variance  (MANOVA)  was  used  primarily  as 
"protection*  from  an  inflated  alpha  levd  on  die 
univariate  tests  comparing  iSA  and  SBI  com¬ 
posite  ratings  and  technical  performance  mea¬ 
sures  between  the  groups.  Although  statistically 
significant  differences  are  not  unexpected,  pri¬ 
marily  because  of  the  large  sample  size,  these 
differences  may  not  have  a  meaningful  impact 
on  the  likelihood  of  success  in  the  different 
phases  of  SIDP.  Thus,  logistic  r^ession 
analyses  were  conducted  to  determine  the  rdative 
influence  of  the  SIDP  and  performance  variables 
on  the  probability  of  dichotomous  (i.e,  "pass- 
fail”)  SIDP  outcomes. 

RESULTS 

Correlations  were  computed  between  each  of 
the  dimensions  rated  in  the  SIDP.  High  correla¬ 
tions  among  PSA  ratings  and  among  SBI  ratings 
(see  Table  I)  suggested  that  composite  variables 
for  the  PSA  and  SBI  might  be  needed.  Separate 
factor  analyses  (principle  components  and 
orthogonal  rotation)  were  conducted  for  the  PSA 
and  SBI  dimension  ratings.  Only  1  factor  was 
found  for  the  PSA;  scores  on  each  dimension 
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were  avn-aged  for  a  PSA  Composite  score.  Two 
factors  were  found  for  the  SBI  ratings:  a 
knowledge  factor  made  up  of  the  2  knowledge 
rating  dimensions  (Knowledge  of  Supervisory 
Role  Performance  and  Organizational 
Knowledge)  and  a  skill  factor  made  up  of  the 
remaining  seven  rating  dimensions.  Because 
nttings  on  the  knowledge  dimensions  are  based 
on  answers  to  questions  rather  than  p^ormance 
in  role-play  scmarios,  there  was  some  concern 
diat  die  2  factors  reflected  method  variance 
rathff  than  separate  dimensions.  Thus,  all  items 
were  included  in  an  item  analysis  using  die 
rdiabilities  (nocedure  in  SPSS*  (Nye,  1990). 
Item  statistics  suggested  diat  the  2  SBI  factors 
should  be  kept  separate  for  subsequoit  statistical 
analyses.  Ratings  for  each  respective  factor  were 
sununed  to  create  a  SBI  Knowledge  Composite 
and  a  Skill  Conqxisite.  Corrdations  between  the 
composite  variables  and  the  technical 
performance  variables  are  shown  in  Table  2; 
descriptive  statistics  for  the  variables  for  different 
group  comparisons  are  shown  in  Table  3. 

Referral  to  the  SBI 

Analysis  of  variance  (ANOVA)  was  used  for 
comparing  mean  differences  between  the  Referred 
and  Not  Referred  groups.  Only  the  Performance 
Rating  and  Months  FPL  Experience  were 
included  since  the  PSA  composite  operationally 
defines  wdio  will  be  referr^  to  the  SBI  and 
would  be  expected  to  be  lower  for  the  Not 
Referred  group  than  the  Referred  group.  The 
Referred  and  Not  Referred  groups  had 
significandy  different  mean  Performance  Ratings 
(F=  101.81,  4C=  1,983,  /X.OOl)  but  wwe  not 
significandy  different  on  Months  FPL  Experience 
(F<1.0). 

The  probability  of  being  referred  to  the  SBI 
was  estimated  using  the  SPSS*  logistic  regres¬ 
sion  procedure  (NoruSis,  1990).  In  the  first 
r^ession,  the  PSA  Composite,  Performance 
Ratings,  and  Months  FPL  Experience  were 
entered  into  the  regression  equation  using  the 
forward  st^wise  method.  The  second  regression 
excluded  the  PSA  Composite.  It  was  felt  that 
comparison  of  the  2  regressions  would  help  to 
identify  the  impact  of  the  peripheral  performance 
variables  on  die  probability  of  referral  to  the 


SBI.  Results  of  the  first  regression  showed  that 
only  the  PSA  Composite  and  Performance  Rating 
variables  predicted  the  referral  decision.  The 
goodness-of-fit  was  tested  using  -2  times  the  log 
likelihood  (-2  LL)  of  the  model  and  indicated  a 
good  fit  to  the  data  (-2  LL,  x'(l,982)=555.42, 
p=l).  The  overall  classification  rate  was 
89.85%.  When  the  PSA  Composite  was  excluded 
from  the  equation.  Performance  Rating  was  still 
identified  as  a  predictor.  However,  the  fit  to  die 
data  was  somewhat  poor  (-2  LL, 
x\l,983)=986.10,  />=.47)  and  all  cases  were 
predicted  to  be  referred  to  the  SBI. 

Promotability 

A  multivariate  analysis  of  variance  test  was 
used  to  examine  mean  differences  on  the  PSA 
composite.  Performance  Rating,  and  Months  FPL 
Experience  betweai  those  who  were  placed  on 
the  EFC  list  (EFQ  and  diose  who  were  not  {Not 
EFQ.  SBI  variables  were  excluded  from  this 
analysis  because  the  combination  of  rating  scores 
on  the  dimensions  operationally  determines  who 
is  placed  on  the  EFC  list.  The  multivariate  test 
was  not  significant. 

Again,  2  logistic  r^ession  analyses  were 
conducted.  The  first  regression  equation  in¬ 
cluded  the  SBI  Knowledge  and  Skill  Com¬ 
posites,  as  well  as  the  PSA  Composite,  Perfor¬ 
mance  Rating,  and  Months  FPL  Experience. 
Only  the  SBI  Knowledge  and  Skill  Composites 
predicted  the  promotability  decision.  The  good¬ 
ness-of-fit  was  tested  using  -2  times  the  log 
likelihood  (-2  LL)  of  the  model  and  indicated  a 
good  fit  to  the  data  (-2  LL,  x\l, 742)  =195.98, 
p=\).  The  overall  classification  was  95.03%. 
Given  that  the  SBI  ratings  are  used  to  opera¬ 
tionally  determine  assignment  to  the  EFC  list, 
this  finding  is  not  surprising.  The  second 
regression  analysis  excluded  the  SBI  composite 
variables;  the  PSA  Composite  and  technical 
performance  variables  did  not  enter  into  the 
prediction  of  promotability  status. 

Seleaion 

The  dependent  variables  included  in  the 
MANOVA  were:  SBI  Knowledge  Composite, 
SBI  Skill  Composite,  PSA  Composite,  Perfor- 
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mance  Rating,  and  Months  FPL  Experience.  The 
MANOVA  test  was  statistically  significant  (Pi- 
llai’s  Trace  criterion,  F=4.00,  <^=5,543 
p<.001).  ANOVA  tests  demonstrated  signifi¬ 
cantly  diffn’ent  group  means  on  the  PSA  Com¬ 
posite  (F=  13.56,  d/=l,547,/?<.001),  the  SBI 
Knowl^ge  Composite  (F=4.67,  df=  1,541, 
/j<.05),  and  Performance  Rating  (F=4.74, 
4r=  1,547,  p<  .05). 

The  logistic  regression  analysis  included  all 
the  variables  used  in  the  MANOVA.  The  results 
showed  that  only  the  PSA  Composite  entered 
intothe  equation  for  predicting  selection  for  a 
supervisory  position.  Assessment  of  the  good- 
ness-of-fit  of  the  model  indicated  a  poor  fit  to  the 
data  (x'(l,547)=605.78,  /)=.04).  Although  the 
overall  classification  rate  was  74.86%,  the  model 
predicted  that  no  one  had  been  selected  for  a 
supervisory  position. 

DISCUSSION 

The  results  of  the  analysis  of  the  referral 
variable  identified  a  significant  difference  bet¬ 
ween  the  Referred  and  Not  Referred  groups  on 
the  PSA  Pedbrmance  Ratings.  In  addition,  the 
PSA  composite  and  p^ormance  rating  predicted 
referral  to  the  SBI.  Although  the  applicant’s 
performance  rating  is  not  considered  in  the  SIDP 
as  part  of  the  decision  to  refer  to  the  SBI,  it  may 
contain  information  that  is  redundant  with  peer 
and  supervisory  ratings  of  the  applicant  on  the 
PSA  dimensions.  In  fact,  performance  ratings  did 
show  small  but  significant  correlations  with  each 
of  the  PSA  dimension  ratings.  Thus,  the  perfor¬ 
mance  rating  may  reflect  similar  or  additional 
information  on  the  differences  among  applicants 
in  supervisory  and  technical  abilities. 

The  two  groups  considered  in  the  analysis  of 
promotability  were  not  significantly  different  on 
the  PSA  Composite  or  technical  performance 
variables.  Of  the  independent  variables  used  in 
the  logistic  regression  analysis,  only  the  SBI 
composite  predicted  promotability.  This  is 
perhaps  as  it  should  be,  since  the  SBI  is  meant  to 
identify  more  keenly  the  skills  rated  by 
supervisors  and  peers  in  the  PSA  (excluding 
te^nical  competence)  and  additional  important 
oimpetencies  as  identified  in  a  study  of  FAA 


air  traffic  control  first-line  supervisor 
competencies  (Myers  &  Stutzman,  1991). 
Because  the  SBI  is  conducted  with  interviewers 
who  do  not  know  the  candidate  and  the  content 
of  the  scenarios  does  not  include  aspects  of  the 
air  traffic  controller’s  operational  job,  technical 
performance  (as  measured  by  performance 
ratings  and  experience)  should  not  play  a  role  in 
performance  at  this  stage.  Of  course,  an 
individual’s  performance  in  the  SBI  may  be 
enhanced  through  other  types  of  work 
experience,  for  example,  prior  staff  positions  or 
experience  as  an  on-the-job-training  or  Academy 
instructor.  These  types  of  experiences  bring  with 
them  opportunities  to  interact  with  others  to 
accomplish  tasks  unlike  those  normally  found  in 
the  air  traffic  controller’s  operational  position, 
and  can  enhance  current  supervisory  skills  and 
the  development  of  new  skills. 

Differences  between  those  who  were  selected 
for  a  supervisory  position  and  those  who  wo'e 
not  were  found  for  the  PSA  Composite,  SBI 
Knowledge  Composite,  and  Performance  Rating 
variables.  Although  we  cannot  know  for  certain 
which  factors  the  selecting  official  is  emphasizing 
in  making  a  selection  decision  based  on  this 
analysis,  it  does  appear  that  several  factors  are 
being  considered  in  the  selection  of  supervisors. 
The  measures  for  which  there  were  significant 
mean  differences  encompassed  the  5  most 
frequently  mentioned  competencies  important  to 
task  accomplishment  (communication,  leadership, 
and  interpersonal  skills;  knowledge  of 
supervisory  role  performance,  and  technical 
competence)  identified  by  incumb^t  air  traffic 
control  supervisors  (Myers  &  Stutzman,  1991). 
Thus,  selection  officials  appear  to  be  choosing 
individuals  for  supervisory  positions  in  line  with 
incumbent  perc^tions  of  important  competencies 
in  performing  foe  supervisory  job.  Given  foe 
various  sources  of  foe  ratings  (peers,  supervisors, 
and  trained  assessors)  selecting  officials  may  also 
be  considering  multiple  inputs  on  foe  applicant’s 
qualifications  in  making  foe  selection  decision. 
However,  these  results  must  be  considered  in 
light  of  foe  fact  that  only  foe  PSA  Composite 
entered  into  foe  logistic  regression  equation  to 
predict  selection  status  and  foe  model  did  not  fit 
foe  data  well.  The  importance  of  different  factors 
in  attaining  a  supervisory  position  is  still  not  well 
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defined. 

Interviews  with  air  traffic  facility  selecting 
officials  conducted  during  an  evaluation  of  the 
SIDP  noted  that  factors  other  than  SIDP  perfor¬ 
mance,  such  as  diversity  in  work  experiences  and 
referraces  from  applicants’  supervisors,  play  a 
large  part  in  the  selection  decision  (Office  of  the 
Associate  Administrator  for  Human  Resource 
Management,  1991).  Identification  of  important 
factors  in  the  selection  decision  is  needed  as  part 
of  the  validation  of  the  SIDP.  In  addition,  collec¬ 
tion  of  information  on  applicant  experiences, 
education,  and  other  biodata  might  suggest  what 
additional  types  of  factors  may  relate  to  success¬ 
ful  performance  in  SIDP.  This  type  of  informa¬ 
tion  could  be  provided  to  af^licants  to  improve 
their  chances  for  success  in  the  selection  pro¬ 
gram,  as  well  as  improving  their  effectiveness  as 
a  supervisor. 

Obviously,  this  study  did  not  span  the  do¬ 
main  of  possible  predictors  of  success  in  SIDP 
and  selection  as  a  first-line  supervisor.  Future 
research  in  areas  such  as  biodata,  achievement 
motivation,  and  personality  factors  is  needed  to 
contribute  to  the  theoretic^  base  for  managerial 
potential  and  assist  in  improving  the  operation 
of  SIDP. 
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Table  1 

Correlations  among  technical  performance  measures,  PSA  ratings,  and  SBI  ratings 
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Table! 

Correlations  between  performance  variables 


1)  PSA  Composite 


2)  SBI  Knowledge 

.13* 

3)  SBI  Skill 

.07 

.44* 

4)  Performance  Rating 

.ir 

.00 

.03 

S)  Months  to  FPL 

-.04 

-.03 

-.01 

.00 

Measures 

1 

2 

3 

4 

*p  ^  .01  (JV  =  743) 
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Table  3A 

Mean  scores  by  referral  status 

Referred  to  SBI 
(N  =  746) 

Not  Referred 
(N  =  239) 

Measures 

M 

SD 

M 

SD 

P^onnance  Rating 

Months  FPL  Experience 

4.00 

44.46 

.67 

13.75 

3.52 

43.56 

.57 

14.79 

Table  3B 

Mean  scores  by  promotability  status 

Eligible  for  Consideration 
(N  =  549) 

Not  Eligible 
(N  =  197) 

Measures 

M 

SD 

M 

SD 

PSA  Composite 

64.13 

10.80 

63.07 

11.59 

Performance  Rating 

Months  FPL  Experience 

4.01 

44.46 

.67 

13.75 

3.98 

44.52 

.67 

14.04 

Table  3C 

Mean  scores  by  selection  status 

Selected 
{N  =  138) 

Not  Selected 
(N  =  410) 

Measures 

M 

SD 

M 

SD 

PSA  Composite 

67.03 

11.38 

63.16 

10.44 

SBI  Knowledge  Composite 
SBI  Skill  Composite 

8.09 

30.61 

1.45 

2.90 

7.91 

30.33 

.67 

2.84 

Performance  Rating 

Months  FPL  Experience 

4.12 

45.20 

.70 

13.03 

3.97 

44.18 

.66 

13.83 

*u.s. 
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